-- Logs begin at Thu 2022-06-16 13:58:43 UTC, end at Thu 2022-06-16 15:19:40 UTC. -- Jun 16 14:39:13.719512 dv-cinder-837165-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 14:39:13.733246 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 14:39:13 2022] *** Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: nodename: dv-cinder-837165-3-iscsi Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: machine: x86_64 Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: clock source: unix Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: pcre jit disabled Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: detected number of CPU cores: 4 Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: current working directory: / Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: detected binary path: /usr/bin/uwsgi-core Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: your processes number limit is 47680 Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: your memory page size is 4096 bytes Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: detected max file descriptor number: 2048 Jun 16 14:39:13.736556 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: lock engine: pthread robust mutexes Jun 16 14:39:13.738007 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: thunder lock: enabled Jun 16 14:39:13.738007 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 14:39:13.738007 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 14:39:13.813897 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: Python main interpreter initialized at 0x55af115279c0 Jun 16 14:39:13.813897 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: python threads support enabled Jun 16 14:39:13.813897 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: your server socket listen backlog is limited to 100 connections Jun 16 14:39:13.813897 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: your mercy for graceful operations on workers is 90 seconds Jun 16 14:39:13.814488 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 14:39:13.814609 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: *** Operational MODE: preforking *** Jun 16 14:39:13.815039 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: *** uWSGI is running in multiple interpreter mode *** Jun 16 14:39:13.815039 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: spawned uWSGI master process (pid: 86680) Jun 16 14:39:13.815302 dv-cinder-837165-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 14:39:13.816211 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: spawned uWSGI worker 1 (pid: 86693, cores: 1) Jun 16 14:39:13.816984 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: spawned uWSGI worker 2 (pid: 86694, cores: 1) Jun 16 14:39:13.816984 dv-cinder-837165-3-iscsi devstack@keystone.service[86680]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 14:39:21.036876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:39:21.036876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: from cryptography.utils import int_from_bytes Jun 16 14:39:21.038176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:39:21.038176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: from cryptography.utils import int_from_bytes Jun 16 14:39:21.102841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 14:39:21.103132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 14:39:21.103408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] command line args: [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 14:39:21.103783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 14:39:21.104052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ================================================================================ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 14:39:21.104409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] admin_token = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.104755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config_dir = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.105088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.105372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] config_source = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.105647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] debug = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.105949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.106483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.106767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.107139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.107523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.108016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.109018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_config_append = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.109018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.109018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_dir = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.109364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.109652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.110070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.110453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.110830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.111133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.111434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.111810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.112187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.112548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.112948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.113310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.113665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.114033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.114409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.114811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.115436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.115739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] publish_errors = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.116162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.116522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.116866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.117311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.117669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.118033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] standard_threads = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.118393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.118755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.119052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.119352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_journal = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.119646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_json = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.119944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_stderr = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.120240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] use_syslog = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.120601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:21.121033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.121395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.121822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.122194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.122568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.122935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.123307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.123665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.124032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.124417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.124798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.125191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.125562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.125923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.126295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.126660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.127026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.127401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.127782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.128154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.128513 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.128924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.129322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.129695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.130062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.130415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.130774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.131124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.131491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.131877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.132252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.133147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.133506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.133876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.134374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.134858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.135284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.135763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.136125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.136540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.136998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.137430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.137859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.138129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.138389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.138735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.138996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.139295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.139575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.139810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.140058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.140319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.140819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.141102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.141347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.141589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.141865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.142210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.142546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.142863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.143135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.143381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.143582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.143856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.144102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.144339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.144579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.144951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.145399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.145648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.145934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.146247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.146493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.146735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.146977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.147236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.147479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.147720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.147969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.148822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.149188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.149556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.149859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.150250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.150636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.150967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.151362 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.151570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.151759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.152256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.152639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.152820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.153742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.153742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.153742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.154429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.154429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.155998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.155998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.155998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.155998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.157123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.157123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.157123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.157903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.external = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.157903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.158149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.158370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.158673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.password = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.159050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] auth.token = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.159416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.159628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.159895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.160438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.161022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.161399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.161884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.162495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.162495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.162860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.163170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.163378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.163986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.164350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.164712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.164935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.165151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.165464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.165965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.166477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.166689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.167075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.167475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.167835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.168110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.168508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.169011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.169337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.169689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.169987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.170301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.170646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.170965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.171548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.171738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.172031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.172478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.173080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.173360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.173714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.174101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.174535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.183086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.185273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.187157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.188973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.190887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.192740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.194575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.194575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.194575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.194575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.194575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.194575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.194575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.196557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.196557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.196557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.196557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.196557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.197619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.197619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.197619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.198218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.198431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.198647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.198992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.199315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.199715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.200464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.200464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.201468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.201468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.201468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.202267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.202267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.202678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.202678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.203088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.203319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.203576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.203818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.204124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.204374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.204729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.205073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.205286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.205506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.205914 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.206351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.206951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.207553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.207948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.208176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.208456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.208923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.209352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.209771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.210194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.211314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.211671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.212391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.212806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.213288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.213731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.214185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.214732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.215049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.215466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.216172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.216452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.217046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.218509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.218509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.218509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.218509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.220310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.220310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.220310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.220310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.220310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.221530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.221530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.221530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.221530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.222369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.222369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.222369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.222960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.222960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.223394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.223607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.223816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.224038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.224255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.224555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.224958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.225325 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.225599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.225842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.226730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.226957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.227217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.227548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.227820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.228130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.228353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.228615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.228898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.229185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.229435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.229699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.230038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.230297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.230539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.230829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.231079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:21.231312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86693) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 14:39:21.368406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.368740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.369118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.385871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.429338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.429604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.498741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.564023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.564339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.651527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.651770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.652048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.652281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.652503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.652751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.681392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.681645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.681878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.682187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.682455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.682658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.682985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.697188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.710990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.711239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.711430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.711626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.711844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.712060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.712274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86693) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:21.735894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.736158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:39:21.736560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.736871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:39:21.737216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.737482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.737797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.738102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.751219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.751669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.752091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.752913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.763886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.768530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.769015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.769418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.769806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.770171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.770602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.771031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.771380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.771887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.772413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.772791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.802592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.805130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.805520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.811888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.812464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.812875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.823092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.823438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.828291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.828592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.828903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.829190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.829466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.829741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.830014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.830284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.854318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.854789 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.859878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.865012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.865553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.872634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.876984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.882002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.883024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.883479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.884046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.884226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.884504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.910934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.911189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.911502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.911822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.921095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.923312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.923756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.929493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.929868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.930293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.930744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.931243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.949638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.949952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.950223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.950621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.951053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.951422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.951825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.976032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.980075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.984303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.986564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.988932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:21.989303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.989666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:21.999222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.000955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:22.004286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.004581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.005029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.005313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.016401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:22.016717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.017031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.029687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86693) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:22.030153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.030475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.030760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.031054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.031530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.031870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.032267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.032659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.033027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.033339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.033687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.033977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.034264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86693) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:22.075564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WSGI app 0 (mountpoint='') ready in 9 seconds on interpreter 0x55af115279c0 pid: 86693 (default app) Jun 16 14:39:22.079512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db20c-9762-40f0-8c9a-b914516e581a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:22.079810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db20c-9762-40f0-8c9a-b914516e581a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:22.080115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90db20c-9762-40f0-8c9a-b914516e581a None None] PATH_INFO: `/v3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:22.084742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1/1] 10.222.0.72 () {50 vars in 791 bytes} [Thu Jun 16 14:39:22 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:22.599193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:39:22.599193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: from cryptography.utils import int_from_bytes Jun 16 14:39:22.600252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 14:39:22.600252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: from cryptography.utils import int_from_bytes Jun 16 14:39:22.650939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 14:39:22.651178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 14:39:22.651302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] command line args: [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 14:39:22.651578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 14:39:22.651757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ================================================================================ {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 14:39:22.652016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] admin_token = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.652270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] config_dir = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.652458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.652655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] config_source = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.652864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] debug = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.653057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.653308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.653490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.653751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.654002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] list_limit = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] log_config_append = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] log_dir = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] log_file = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.655973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.656106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.656240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.656368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.656611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.656886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.657200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.657441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.657677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.657911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.658167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.658416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.658700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.659230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.659416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] publish_errors = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.659662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.659880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.660064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.660317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.660556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.660796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] standard_threads = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.661075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.661326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.661506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.661761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] use_journal = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.661954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] use_json = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.662167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] use_stderr = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.662325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] use_syslog = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.662571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 14:39:22.662840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.663082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.663351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.664238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.664600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.664877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.665138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.665384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.665642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.665900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.666161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.666402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.666659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.666896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.667154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.667407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.667651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.667924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.668186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.668439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.668683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.668969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.669251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.669509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.669825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.670069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.670304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.670540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.670783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.671040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.671297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.671539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.671775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.672022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.672276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.672520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.672772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.673037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.673282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.673521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.673756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.673992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.674226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.674462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.674702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.675096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.675338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.675583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.675819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.676062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.676304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.676558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.676796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.677066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.677307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.677552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.677795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.678035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.678271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.678524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.678772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.679012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.679263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.680079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.680365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.680605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.680875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.681117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.681402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.681648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.681904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.682164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.682402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.682640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.682881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.683134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.683390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.683632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.683882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.684124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.684362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.684599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.684931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.685103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.685380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.685628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.685870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.686110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.686361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.686601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.686839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.687076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.687311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.687544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.687775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.688012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.688243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.688489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.688725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.689035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.689287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.689550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.689820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.690064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.690296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] auth.external = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.690530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.690833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.691068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.691308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] auth.password = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.691540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] auth.token = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.691785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.692022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.692274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.692507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.692756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.693045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.693280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.693518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] credential.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.693769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.694022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.694272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.694510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.694754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.695524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.695795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.696238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.696570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.696867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.697161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.697415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.697692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.697942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.698202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.698441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.698683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.698933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.699185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.699462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.699713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.699957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.700215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.700458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.700716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.700982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.701241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.701481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.701730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.701983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.702222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.702460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.702764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.703058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.703357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.703687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.703954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.704193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.704448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.704684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.704960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.705216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.705471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.705730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.705981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.706271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.706513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.706753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.706987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.707228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.707460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.707700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.707978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.708171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.708424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.708658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.708936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.709189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.709425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.709673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.709923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.710156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.710397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.711184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.711494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.711739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.711994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.712234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.712490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.712747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.713027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.713262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.713502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.713755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.714004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.714245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.714483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.714720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.714952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.715196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.715458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.715694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.715942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.716192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.716439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.716676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.716939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.717181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.717418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.717658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.717889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.718139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.718386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.718637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.718886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.719135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.719366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.719615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.719853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.720092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.720331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.720636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.720897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.721077 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.721326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.721583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.721817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.722066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.722303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.722538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.722776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.723024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.723271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.723503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.723740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.723982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.724236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.724487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.724727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.725011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.725270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.725522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.725774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.726001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.726278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.726514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] role.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.726766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] role.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.727001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.727251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.727502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.727752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.728002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.728252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.728500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.728746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.729024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.729263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.729513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.729762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.730012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.730263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.730713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.730951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.731211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.731464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.731715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.731968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.732206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.732624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.732908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.733351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.733481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.733593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.734021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.734169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.734338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.734589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.734831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.735069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.735303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.735540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.735782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.736032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.736273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.736529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.736779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.737048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.737298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.737544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.737795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.738035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.738276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.738515 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.738778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.739030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.739265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.739510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.739823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.740239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.connection = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.740606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.740909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.741172 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.741671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.742112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.742514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.742770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.743012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.743265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.743509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.743758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.743999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.744254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.744496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.744744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.745003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.745282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.745565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 14:39:22.745855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86694) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 14:39:22.776723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.776998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.777221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.877599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.898267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.898525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.940966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.983227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:22.983524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.033738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.033987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.034160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.034482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.034668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.034821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.050665 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.050919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.051264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.051444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.051583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.051939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.051939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.061061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.064410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.064762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.064936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.065067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.065297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.065513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.065712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86694) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 14:39:23.078534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.078791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:39:23.079080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.079281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 14:39:23.079554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.079824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.080088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.080350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.094222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.094516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.094769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.095062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.104424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.108170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.108452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.108742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.109056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.109338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.109598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.109892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.110166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.110446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.110714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.110977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.138528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.140119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.140397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.146349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.146649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.146940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.155319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.155620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.160358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.160668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.160984 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.161267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.161539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.161842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.162141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.162408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.182748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.183073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.188045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.192099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.192549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.198425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.202353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.206092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.206398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.206685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.206967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.207256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.207548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.227046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.227358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.227644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.227910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.236266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.238083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.238377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.241525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.242377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.242663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.242956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.243388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.256992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.257297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.258144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.258424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.258704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.258976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.259251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.278753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.282764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.286362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.288089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.290427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.290877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.291163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.298795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.300496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.304021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.304349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.304648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.304954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.315168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.315484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.315771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.323938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86694) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 14:39:23.324243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.324529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.324806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.325105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.325413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.325739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.326016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.326285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.326552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.326945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.327230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.327508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.327771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86694) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 14:39:23.369965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WSGI app 0 (mountpoint='') ready in 10 seconds on interpreter 0x55af115279c0 pid: 86694 (default app) Jun 16 14:39:30.615771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b03a-74ef-4b1a-98c7-6e4f5e0e1e7b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:30.616321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b03a-74ef-4b1a-98c7-6e4f5e0e1e7b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:30.617102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3b03a-74ef-4b1a-98c7-6e4f5e0e1e7b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:30.619064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 2/2] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:39:30.628193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:30.628919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:30.628919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:30.700760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: INFO dbcounter [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] Registered counter for database keystone Jun 16 14:39:30.723953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG oslo_db.sqlalchemy.engines [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86694) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 14:39:30.731109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writer thread running {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 14:39:30.928757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG passlib.handlers.bcrypt [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86694) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 14:39:30.929240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG passlib.handlers.bcrypt [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86694) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 14:39:30.987085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:30.988755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:31.140223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1/3] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 14:39:31.290721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3b03a-74ef-4b1a-98c7-6e4f5e0e1e7b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:31.372414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: INFO dbcounter [None req-7df3b03a-74ef-4b1a-98c7-6e4f5e0e1e7b None None] Registered counter for database keystone Jun 16 14:39:31.394350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG oslo_db.sqlalchemy.engines [None req-7df3b03a-74ef-4b1a-98c7-6e4f5e0e1e7b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86693) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 14:39:31.401292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writer thread running {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 14:39:31.534368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3b03a-74ef-4b1a-98c7-6e4f5e0e1e7b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:31.539525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fdc93-ff9f-41bb-9c46-6c9ec31f9517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:31.540219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fdc93-ff9f-41bb-9c46-6c9ec31f9517 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:31.540502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fdc93-ff9f-41bb-9c46-6c9ec31f9517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:31.540777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fdc93-ff9f-41bb-9c46-6c9ec31f9517 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:31.550118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fdc93-ff9f-41bb-9c46-6c9ec31f9517 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:31.649539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING keystone.common.rbac_enforcer.enforcer [None req-992fdc93-ff9f-41bb-9c46-6c9ec31f9517 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 14:39:31.650315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fdc93-ff9f-41bb-9c46-6c9ec31f9517 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:31.661581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 3/4] 10.222.0.72 () {58 vars in 1324 bytes} [Thu Jun 16 14:39:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 14:39:31.668213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:31.684651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad734-bb3c-40c4-9e28-da6334797338 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:31.689709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d37b0-c68f-4717-972b-ce42f6ec4a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:31.690891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d37b0-c68f-4717-972b-ce42f6ec4a2c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:31.691183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d37b0-c68f-4717-972b-ce42f6ec4a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:31.691464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d37b0-c68f-4717-972b-ce42f6ec4a2c admin admin] PATH_INFO: `/v3/projects/1304e58d14b64959b1e4ffc146293183` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:31.693486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d37b0-c68f-4717-972b-ce42f6ec4a2c admin admin] RBAC: Authorizing `identity:get_project(project_id=1304e58d14b64959b1e4ffc146293183)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:31.748799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b27d37b0-c68f-4717-972b-ce42f6ec4a2c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 14:39:31.749170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d37b0-c68f-4717-972b-ce42f6ec4a2c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:31.754183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 2/5] 10.222.0.72 () {56 vars in 1207 bytes} [Thu Jun 16 14:39:31 2022] GET /identity/v3/projects/1304e58d14b64959b1e4ffc146293183 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:39:33.953053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f541-c6e4-4f2e-8386-d13a10e535c4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:33.953862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f541-c6e4-4f2e-8386-d13a10e535c4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:33.953862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f541-c6e4-4f2e-8386-d13a10e535c4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:33.958197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 4/6] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:39:33.965166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb38dc-88fa-4a34-9826-4b0bcdbce3e0 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:33.965731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb38dc-88fa-4a34-9826-4b0bcdbce3e0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:33.965731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb38dc-88fa-4a34-9826-4b0bcdbce3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:33.986467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-38cb38dc-88fa-4a34-9826-4b0bcdbce3e0 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:33.988092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38cb38dc-88fa-4a34-9826-4b0bcdbce3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:33.996456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 3/7] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 14:39:34.021050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f541-c6e4-4f2e-8386-d13a10e535c4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:34.034815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f541-c6e4-4f2e-8386-d13a10e535c4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:34.040088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3c885-c171-40f2-82ff-99c3945b38b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:34.040755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3c885-c171-40f2-82ff-99c3945b38b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:34.041078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3c885-c171-40f2-82ff-99c3945b38b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:34.041360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3c885-c171-40f2-82ff-99c3945b38b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:34.050931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3c885-c171-40f2-82ff-99c3945b38b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:34.052908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3c885-c171-40f2-82ff-99c3945b38b2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:34.064041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 5/8] 10.222.0.72 () {58 vars in 1324 bytes} [Thu Jun 16 14:39:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 14:39:34.070183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb38dc-88fa-4a34-9826-4b0bcdbce3e0 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:34.080190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb38dc-88fa-4a34-9826-4b0bcdbce3e0 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:34.085076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b3ca5-2dc9-4d52-8fc3-4108aac29bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:34.086036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3ca5-2dc9-4d52-8fc3-4108aac29bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:34.086323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3ca5-2dc9-4d52-8fc3-4108aac29bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:34.086601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3ca5-2dc9-4d52-8fc3-4108aac29bb4 admin admin] PATH_INFO: `/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:34.088207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b3ca5-2dc9-4d52-8fc3-4108aac29bb4 admin admin] RBAC: Authorizing `identity:get_user(user_id=b8adc1b6aec146cdaed3b9ac9a211c0a)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:34.090128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b3ca5-2dc9-4d52-8fc3-4108aac29bb4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:34.091878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 4/9] 10.222.0.72 () {56 vars in 1198 bytes} [Thu Jun 16 14:39:34 2022] GET /identity/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:36.521339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f052a-d323-4319-a132-6264dd1a11f9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:36.522159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f052a-d323-4319-a132-6264dd1a11f9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:36.522298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f052a-d323-4319-a132-6264dd1a11f9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:36.524514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 6/10] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:39:36.531013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94b36-f3d3-4e0e-b4ad-643a8a4c19ed None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:36.531803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94b36-f3d3-4e0e-b4ad-643a8a4c19ed None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:36.531969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a94b36-f3d3-4e0e-b4ad-643a8a4c19ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:36.553521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f2a94b36-f3d3-4e0e-b4ad-643a8a4c19ed None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:36.554927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f2a94b36-f3d3-4e0e-b4ad-643a8a4c19ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:36.569476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de76ee-806e-4377-8f88-aa52bec4937e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:36.569919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de76ee-806e-4377-8f88-aa52bec4937e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:36.570383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de76ee-806e-4377-8f88-aa52bec4937e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:36.573063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 7/11] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:39:36.582353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:36.582949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:36.583450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:36.660127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG passlib.handlers.bcrypt [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86693) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 14:39:36.660666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG passlib.handlers.bcrypt [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86693) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 14:39:36.692813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 5/12] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:39:36.715418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:36.718081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:36.733056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94b36-f3d3-4e0e-b4ad-643a8a4c19ed None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:36.781115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:36.878504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 8/13] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:39:36.905277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a94b36-f3d3-4e0e-b4ad-643a8a4c19ed None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:36.916037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:36.967167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:36.968254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:36.968741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:36.969257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:37.081110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d710f2-649e-4d43-8925-e11121e9dba9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:37.086687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:37.099677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:37.101825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:37.149852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:37.155340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:37.157100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:37.157518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:37.157995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:37.188288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:39:37.204816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:37.206062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:39:37.207032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 9/14] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:39:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:39:37.215457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:37.288213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 6/15] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:39:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:39:37.296059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:37.360520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2cfd8-c525-4242-a787-ff990c3a52f1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:37.406475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:37.407280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:37.407647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:37.408025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:37.409092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:37.409544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:37.411791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:37.430393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 10/16] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:39:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:37.441435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:37.465273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ff8e-93fd-47b7-8c59-abc363e3e802 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:37.487368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:37.505344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:37.529757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:37.531166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:37.532135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:37.532135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:37.551926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:37.554728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:39:37.586270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:37.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:37.589930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:39:37.589930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:39:37.589930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:39:37.589930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:39:37.589930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 7/17] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:39:37.595421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:37.616237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9c160-1190-48e9-a952-9811efcbc389 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:37.671537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:37.673850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:37.674352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:37.675054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] PATH_INFO: `/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:37.678022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] RBAC: Authorizing `identity:get_user(user_id=b8adc1b6aec146cdaed3b9ac9a211c0a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:37.681737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:37.684980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 11/18] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:39:37 2022] GET /identity/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:39:37.691144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:37.763459 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34100a6e-071b-466a-96b1-47bd161ec0da None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:37.818279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:37.823084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:37.824254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:37.824760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:37.825303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:37.826670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:37.827267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3f319-a764-4a3c-9fcf-aa10d9d64575 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:37.827434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:37.830890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:37.863587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 8/19] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:37.872078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5841526-7923-485f-8dba-5a11005eae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:37.872708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841526-7923-485f-8dba-5a11005eae38 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:37.873232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841526-7923-485f-8dba-5a11005eae38 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:37.873488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841526-7923-485f-8dba-5a11005eae38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:37.874958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5841526-7923-485f-8dba-5a11005eae38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:37.876633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5841526-7923-485f-8dba-5a11005eae38 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:37.879869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 12/20] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:39:37.888683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:37.964268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:38.080027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f095a-d111-4d0c-a48d-356afc7091f1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:38.122676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f9cb4-bc76-4e55-b445-70d1eb5ef3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:38.123287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f9cb4-bc76-4e55-b445-70d1eb5ef3b3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:38.123640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f9cb4-bc76-4e55-b445-70d1eb5ef3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:38.123997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f9cb4-bc76-4e55-b445-70d1eb5ef3b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:38.124888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f9cb4-bc76-4e55-b445-70d1eb5ef3b3 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.domain.id=default, user.id=b8adc1b6aec146cdaed3b9ac9a211c0a) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:38.125263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f9cb4-bc76-4e55-b445-70d1eb5ef3b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:38.126933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f9cb4-bc76-4e55-b445-70d1eb5ef3b3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:38.136576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 9/21] 10.222.0.72 () {56 vars in 1322 bytes} [Thu Jun 16 14:39:37 2022] GET /identity/v3/role_assignments?user.id=b8adc1b6aec146cdaed3b9ac9a211c0a&scope.domain.id=default&role.id=c21668b08994436ca32efab183c5ca03 => generated 233 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:41.605597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c72709-16fc-435e-9c85-bb16f02d7ecb None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:41.607133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c72709-16fc-435e-9c85-bb16f02d7ecb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:41.607938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c72709-16fc-435e-9c85-bb16f02d7ecb None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:41.610920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 13/22] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:41.618893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64dfcb-b529-428d-962b-349e410b9c9d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:41.620404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64dfcb-b529-428d-962b-349e410b9c9d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:41.621214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64dfcb-b529-428d-962b-349e410b9c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:41.661051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-4d64dfcb-b529-428d-962b-349e410b9c9d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:41.664050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4d64dfcb-b529-428d-962b-349e410b9c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:41.801956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 10/23] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:41.846324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c72709-16fc-435e-9c85-bb16f02d7ecb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:41.986909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96195174-8f8b-4e63-972c-c03c6be559c7 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:41.987416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96195174-8f8b-4e63-972c-c03c6be559c7 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:41.987919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96195174-8f8b-4e63-972c-c03c6be559c7 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:41.990568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 11/24] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:41.999761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98a05-2056-4704-acf9-ea06011a8ccc None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:42.000702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98a05-2056-4704-acf9-ea06011a8ccc None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:42.001552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98a05-2056-4704-acf9-ea06011a8ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:42.057688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fcd98a05-2056-4704-acf9-ea06011a8ccc None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:42.058840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c72709-16fc-435e-9c85-bb16f02d7ecb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:42.060595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fcd98a05-2056-4704-acf9-ea06011a8ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:42.140530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:42.141945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:42.142396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:42.143050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:42.143983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:42.145569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:42.145735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:42.289470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 12/25] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:42.290976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302999-5392-4aec-a65c-ab72e1d68105 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:42.291332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302999-5392-4aec-a65c-ab72e1d68105 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:42.291624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302999-5392-4aec-a65c-ab72e1d68105 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:42.293107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 13/26] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:42.302829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880002aa-78d3-4ff4-abec-599c79e5fec8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:42.302829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880002aa-78d3-4ff4-abec-599c79e5fec8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:42.303267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880002aa-78d3-4ff4-abec-599c79e5fec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:42.311097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 14/27] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Jun 16 14:39:41 2022] POST /identity/v3/roles => generated 219 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:39:42.337279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:42.338730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-880002aa-78d3-4ff4-abec-599c79e5fec8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:42.340062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-880002aa-78d3-4ff4-abec-599c79e5fec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:42.420514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:42.582161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 14/28] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:42.644966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee180d23-7a2b-4541-b32b-17ce569a6758 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:42.659356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880002aa-78d3-4ff4-abec-599c79e5fec8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:42.789719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:42.791145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:42.791646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:42.792152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:42.831772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:39:42.847516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:39:42.862017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:42.863879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:42.865503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:39:42.869949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 15/29] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 535 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:39:42.873806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880002aa-78d3-4ff4-abec-599c79e5fec8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:42.883781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:42.930454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793f24-f6f1-486a-ab7f-1eb441c78b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:42.932727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793f24-f6f1-486a-ab7f-1eb441c78b15 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:42.934985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793f24-f6f1-486a-ab7f-1eb441c78b15 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:42.936623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793f24-f6f1-486a-ab7f-1eb441c78b15 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:42.937670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793f24-f6f1-486a-ab7f-1eb441c78b15 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:42.941417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793f24-f6f1-486a-ab7f-1eb441c78b15 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:43.101532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:43.126258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53e6f0-77bf-4843-9d20-3f72c30e792a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:43.211054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:43.212412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.213085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.213700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.215186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:43.215950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:43.218947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:43.258316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 16/30] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:43.267099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:43.419327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 15/31] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Jun 16 14:39:42 2022] POST /identity/v3/roles => generated 225 bytes in 761 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:39:43.427632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46e35f-d63e-4e43-8121-dc727b3749de None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.428328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46e35f-d63e-4e43-8121-dc727b3749de None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.429525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46e35f-d63e-4e43-8121-dc727b3749de None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.432184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 16/32] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:43.441727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fe254-ea4c-408f-8c45-eeb009a215e6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.442171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fe254-ea4c-408f-8c45-eeb009a215e6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.442652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fe254-ea4c-408f-8c45-eeb009a215e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.489447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b22fe254-ea4c-408f-8c45-eeb009a215e6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:43.492595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b22fe254-ea4c-408f-8c45-eeb009a215e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:43.541942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf2c2b-d2ac-4a55-b5f8-d47447730003 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:43.612161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1027e0b-e50c-40f5-ab72-69ff5bf6c170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:43.614346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1027e0b-e50c-40f5-ab72-69ff5bf6c170 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.615192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1027e0b-e50c-40f5-ab72-69ff5bf6c170 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.615816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1027e0b-e50c-40f5-ab72-69ff5bf6c170 None admin] PATH_INFO: `/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.619143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1027e0b-e50c-40f5-ab72-69ff5bf6c170 None admin] RBAC: Authorizing `identity:get_user(user_id=b8adc1b6aec146cdaed3b9ac9a211c0a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:43.621658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1027e0b-e50c-40f5-ab72-69ff5bf6c170 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:43.624346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 17/33] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:39:43 2022] GET /identity/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a => generated 247 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:39:43.626693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d520f7b-21c1-49da-9b97-8095e4fb03f2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.627481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d520f7b-21c1-49da-9b97-8095e4fb03f2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.628255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d520f7b-21c1-49da-9b97-8095e4fb03f2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.637311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 18/34] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:43.639285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d520f7b-21c1-49da-9b97-8095e4fb03f2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:43.677296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:43.680544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 17/35] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:39:43.683808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d6ca-dba4-4d02-8dc5-85edc543f133 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.684643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d6ca-dba4-4d02-8dc5-85edc543f133 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.685371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d6ca-dba4-4d02-8dc5-85edc543f133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.716035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f719d6ca-dba4-4d02-8dc5-85edc543f133 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:43.718753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f719d6ca-dba4-4d02-8dc5-85edc543f133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:43.855442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 18/36] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:43.856969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bcac9-5aee-45a0-8a35-354bf9d79753 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.857322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bcac9-5aee-45a0-8a35-354bf9d79753 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.857994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bcac9-5aee-45a0-8a35-354bf9d79753 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.860715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 19/37] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:43.861642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bcac9-5aee-45a0-8a35-354bf9d79753 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:43.864059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d520f7b-21c1-49da-9b97-8095e4fb03f2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:43.884266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:43.921982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4d93c-8d9a-4718-a3b0-2d25c5efd0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:43.923446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d93c-8d9a-4718-a3b0-2d25c5efd0d1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.923950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d93c-8d9a-4718-a3b0-2d25c5efd0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.924422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d93c-8d9a-4718-a3b0-2d25c5efd0d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.926893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4d93c-8d9a-4718-a3b0-2d25c5efd0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:43.929743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4d93c-8d9a-4718-a3b0-2d25c5efd0d1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:43.932528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 19/38] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:43.934798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bca8eb-03fa-4fec-9e6a-831acd87a4cb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:43.935327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bca8eb-03fa-4fec-9e6a-831acd87a4cb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:43.935797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bca8eb-03fa-4fec-9e6a-831acd87a4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:43.964077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-42bca8eb-03fa-4fec-9e6a-831acd87a4cb None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:43.966981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-42bca8eb-03fa-4fec-9e6a-831acd87a4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:44.034019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bcac9-5aee-45a0-8a35-354bf9d79753 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:44.070875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 20/39] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:44.071678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bca8eb-03fa-4fec-9e6a-831acd87a4cb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:44.096805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:44.115928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:44.118224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:44.118903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:44.118903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:44.133102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:44.134863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:39:44.160680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:44.162112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:44.163089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:39:44.163089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:39:44.163089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:39:44.163089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:39:44.163089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 20/40] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:39:44.163089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:44.234456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bca8eb-03fa-4fec-9e6a-831acd87a4cb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:44.236109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:44.278091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:44.279084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:44.279383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:44.279665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:44.280445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:44.282396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:44.287399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe12495-83bf-41b0-9a29-c134028f9ab6 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:44.298655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 21/41] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Jun 16 14:39:44 2022] POST /identity/v3/roles => generated 223 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:39:44.305108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:44.352089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:44.353972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:44.354451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:44.354916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] PATH_INFO: `/v3/domains/default/users/b8adc1b6aec146cdaed3b9ac9a211c0a/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:44.358356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b8adc1b6aec146cdaed3b9ac9a211c0a, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:44.360931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:44.385967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 21/42] 10.222.0.72 () {58 vars in 1361 bytes} [Thu Jun 16 14:39:44 2022] PUT /identity/v3/domains/default/users/b8adc1b6aec146cdaed3b9ac9a211c0a/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:39:44.387413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:44.439884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:44.529856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002b35c-d464-4712-8f6c-9bdf19d77b00 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:44.615213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:44.616547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802c4f9-6b61-41c2-b64a-5eab205a5058 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:44.617122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:44.617602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:44.618079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:44.619947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:44.621675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:44.623980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 22/43] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:44.634175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:44.651567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:44.677758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:44.678603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:44.678878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:44.681735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:44.681735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:44.681735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:44.682748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:44.706472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 22/44] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:44.714502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:44.889463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb107ad6-e719-4536-b6b5-12957d7f3a72 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:44.931398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8b485-d909-4382-a474-a3e60595d6d7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:44.952468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b15f37-4a5d-4e89-ae79-29ea4447fc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:44.952468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15f37-4a5d-4e89-ae79-29ea4447fc50 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:44.952952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15f37-4a5d-4e89-ae79-29ea4447fc50 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:44.953098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b15f37-4a5d-4e89-ae79-29ea4447fc50 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:44.954450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b15f37-4a5d-4e89-ae79-29ea4447fc50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:44.956580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b15f37-4a5d-4e89-ae79-29ea4447fc50 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:44.991937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049915d8-7d82-46a5-89ea-251a91399238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:44.993082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049915d8-7d82-46a5-89ea-251a91399238 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:44.993481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049915d8-7d82-46a5-89ea-251a91399238 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:44.993934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049915d8-7d82-46a5-89ea-251a91399238 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:44.995242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049915d8-7d82-46a5-89ea-251a91399238 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:44.996695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 23/45] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:39:44 2022] POST /identity/v3/projects => generated 301 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:39:44.998483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049915d8-7d82-46a5-89ea-251a91399238 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:45.027865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:45.076898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 23/46] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:39:44 2022] POST /identity/v3/projects => generated 312 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:39:46.825525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b807-12fa-4e40-88d2-6b3b412a1e8e None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:46.826157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b807-12fa-4e40-88d2-6b3b412a1e8e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:46.826157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b807-12fa-4e40-88d2-6b3b412a1e8e None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:46.827691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 24/47] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:46.833183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44e0f-60df-4d7f-a6d8-36c4bca6622b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:46.833979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44e0f-60df-4d7f-a6d8-36c4bca6622b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:46.834135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a44e0f-60df-4d7f-a6d8-36c4bca6622b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:46.855434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-99a44e0f-60df-4d7f-a6d8-36c4bca6622b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:46.856740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-99a44e0f-60df-4d7f-a6d8-36c4bca6622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:46.937440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 24/48] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:46.961175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b807-12fa-4e40-88d2-6b3b412a1e8e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:46.997042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:47.086958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b807-12fa-4e40-88d2-6b3b412a1e8e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:47.131630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:47.132903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:47.133175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:47.133605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:47.150014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:39:47.159747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:47.161529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:39:47.163136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 25/49] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:39:47.174397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a44e0f-60df-4d7f-a6d8-36c4bca6622b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:47.340945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a44e0f-60df-4d7f-a6d8-36c4bca6622b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:47.370597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:47.391214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c48099-f3c8-49c0-aad6-9eba70a1a9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:47.391910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c48099-f3c8-49c0-aad6-9eba70a1a9db None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:47.392200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c48099-f3c8-49c0-aad6-9eba70a1a9db None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:47.392493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c48099-f3c8-49c0-aad6-9eba70a1a9db None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:47.393441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c48099-f3c8-49c0-aad6-9eba70a1a9db None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:47.393759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c48099-f3c8-49c0-aad6-9eba70a1a9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:47.395185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c48099-f3c8-49c0-aad6-9eba70a1a9db None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:47.409254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 25/50] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:47.415336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:47.516777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37aa5-60b5-4739-8eeb-26c8a0908a10 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:47.517441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37aa5-60b5-4739-8eeb-26c8a0908a10 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:47.517804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d37aa5-60b5-4739-8eeb-26c8a0908a10 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:47.520104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 26/51] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:39:47.527512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df112a9-3a3b-4c66-b117-f92845f050d0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:47.527960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df112a9-3a3b-4c66-b117-f92845f050d0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:47.528376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df112a9-3a3b-4c66-b117-f92845f050d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:47.548942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8df112a9-3a3b-4c66-b117-f92845f050d0 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:47.550217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8df112a9-3a3b-4c66-b117-f92845f050d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:47.568620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:47.580659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874e674-0a07-4ae5-a1de-0c1fed53b8b5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:47.629133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:47.630081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:47.630468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:47.630909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] PATH_INFO: `/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:47.633050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=b8adc1b6aec146cdaed3b9ac9a211c0a)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:47.634845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:47.636770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 26/52] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:39:47 2022] GET /identity/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:39:47.643446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:47.655899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 27/53] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:39:47.682790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df112a9-3a3b-4c66-b117-f92845f050d0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:47.814689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b294e-402a-4197-9acf-e9c21be67ba3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:47.867619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:47.869285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:47.869804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:47.872618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:47.872618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:47.875551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:47.877047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df112a9-3a3b-4c66-b117-f92845f050d0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:47.879272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 27/54] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:39:47.889823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:47.900103 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:47.935063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:47.936256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:47.936544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:47.936829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:47.938402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:47.940259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:47.943276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 28/55] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:47.946303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:47.953167 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:48.092296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dacd9f-892b-4c84-98e8-723580d42334 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:48.132203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489efbf-3cd2-4d0a-88c1-813eff27758a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:48.145747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab66efc-d0b5-4ab0-8870-83ad854e11e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:48.146463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab66efc-d0b5-4ab0-8870-83ad854e11e0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:48.146765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab66efc-d0b5-4ab0-8870-83ad854e11e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:48.147051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab66efc-d0b5-4ab0-8870-83ad854e11e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:48.147987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab66efc-d0b5-4ab0-8870-83ad854e11e0 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.domain.id=default, user.id=b8adc1b6aec146cdaed3b9ac9a211c0a) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:48.148308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab66efc-d0b5-4ab0-8870-83ad854e11e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:48.149920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab66efc-d0b5-4ab0-8870-83ad854e11e0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:48.157834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 28/56] 10.222.0.72 () {56 vars in 1322 bytes} [Thu Jun 16 14:39:47 2022] GET /identity/v3/role_assignments?user.id=b8adc1b6aec146cdaed3b9ac9a211c0a&scope.domain.id=default&role.id=c21668b08994436ca32efab183c5ca03 => generated 530 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:39:48.199417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5cbea-4a26-476f-893e-09aeb3fa577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:48.200591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5cbea-4a26-476f-893e-09aeb3fa577c None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:48.201294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5cbea-4a26-476f-893e-09aeb3fa577c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:48.201813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5cbea-4a26-476f-893e-09aeb3fa577c None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:48.203067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5cbea-4a26-476f-893e-09aeb3fa577c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:48.206118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5cbea-4a26-476f-893e-09aeb3fa577c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:48.257399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 29/57] 10.222.0.72 () {60 vars in 1140 bytes} [Thu Jun 16 14:39:47 2022] POST /identity/v3/projects => generated 298 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:39:50.708807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e63e4-08e2-4a1a-88e0-c8c695b64194 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:50.709735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e63e4-08e2-4a1a-88e0-c8c695b64194 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:50.709735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e63e4-08e2-4a1a-88e0-c8c695b64194 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:50.711385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 29/58] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:50.720025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e3b25-b1be-4dcb-b915-f8cf42b9b06a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:50.720580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e3b25-b1be-4dcb-b915-f8cf42b9b06a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:50.720580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e3b25-b1be-4dcb-b915-f8cf42b9b06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:50.740264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-661e3b25-b1be-4dcb-b915-f8cf42b9b06a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:50.741618 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-661e3b25-b1be-4dcb-b915-f8cf42b9b06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:50.780696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:50.835722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 30/59] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:50.862933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e63e4-08e2-4a1a-88e0-c8c695b64194 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:51.011593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:51.031233 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e63e4-08e2-4a1a-88e0-c8c695b64194 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:51.099888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634cfb8-b8f0-4041-a81d-c1ee7a485a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:51.101387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634cfb8-b8f0-4041-a81d-c1ee7a485a9c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:51.101897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634cfb8-b8f0-4041-a81d-c1ee7a485a9c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:51.102415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634cfb8-b8f0-4041-a81d-c1ee7a485a9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:51.104977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634cfb8-b8f0-4041-a81d-c1ee7a485a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:51.107844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634cfb8-b8f0-4041-a81d-c1ee7a485a9c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:51.111231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 30/60] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:51.120519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e3b25-b1be-4dcb-b915-f8cf42b9b06a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:51.295043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e3b25-b1be-4dcb-b915-f8cf42b9b06a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:51.354656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4715a1-1022-4990-b0cb-927676aa1e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:51.355785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4715a1-1022-4990-b0cb-927676aa1e25 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:51.356323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4715a1-1022-4990-b0cb-927676aa1e25 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:51.356799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4715a1-1022-4990-b0cb-927676aa1e25 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:51.358209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4715a1-1022-4990-b0cb-927676aa1e25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:51.361128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4715a1-1022-4990-b0cb-927676aa1e25 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:51.402613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 31/61] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:39:51 2022] POST /identity/v3/users => generated 275 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:39:55.892801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6cd99-0569-4ae6-b3c6-10b464fadd78 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:55.894342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6cd99-0569-4ae6-b3c6-10b464fadd78 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:55.895166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6cd99-0569-4ae6-b3c6-10b464fadd78 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:55.900128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 31/62] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:55.908228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58966f81-45cb-43c6-85ae-e680a9af67b2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:55.909928 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58966f81-45cb-43c6-85ae-e680a9af67b2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:55.910801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58966f81-45cb-43c6-85ae-e680a9af67b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:55.954902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-58966f81-45cb-43c6-85ae-e680a9af67b2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:55.957866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-58966f81-45cb-43c6-85ae-e680a9af67b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:55.990106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:56.094561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 32/63] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:39:56.148301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f51bc-8a82-4118-98c1-b8a1dbad65d0 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:56.148585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f51bc-8a82-4118-98c1-b8a1dbad65d0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:56.148791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f51bc-8a82-4118-98c1-b8a1dbad65d0 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:56.150263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 32/64] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:56.151621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58966f81-45cb-43c6-85ae-e680a9af67b2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:56.158445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb6957-eec1-4d17-9e21-84afeb59b941 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:56.159110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb6957-eec1-4d17-9e21-84afeb59b941 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:56.159688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb6957-eec1-4d17-9e21-84afeb59b941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:56.206869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2ecb6957-eec1-4d17-9e21-84afeb59b941 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:56.209203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2ecb6957-eec1-4d17-9e21-84afeb59b941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:56.425570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 33/65] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:56.427512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58ce2-2c3b-4c22-9641-edee8e47779b None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:56.428243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58ce2-2c3b-4c22-9641-edee8e47779b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:56.429324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58ce2-2c3b-4c22-9641-edee8e47779b None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:56.431089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 34/66] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:56.437310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676519c-d74f-4722-848a-8a9b60075ce2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:56.437979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676519c-d74f-4722-848a-8a9b60075ce2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:56.438472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676519c-d74f-4722-848a-8a9b60075ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:56.469052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58966f81-45cb-43c6-85ae-e680a9af67b2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:56.471407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2676519c-d74f-4722-848a-8a9b60075ce2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:56.473962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2676519c-d74f-4722-848a-8a9b60075ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:56.531805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:56.533154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:56.533664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:56.534174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:56.547663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:39:56.556280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:56.557251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:39:56.558005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:39:56.565909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 33/67] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:39:56.567252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:56.582953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:56.615177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 35/68] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:39:56.617004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676519c-d74f-4722-848a-8a9b60075ce2 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:56.660659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:56.779858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770479bd-4bc9-4afc-ab4e-d2c7e68e0d54 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:56.816307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676519c-d74f-4722-848a-8a9b60075ce2 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:56.857913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:56.859439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:56.859732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:56.860013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:56.872309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:56.873046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:56.873335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:56.873626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:56.874394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:56.874709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:56.876390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:56.887288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:39:56.905063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:56.906605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:39:56.909240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:39:56.913523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 36/69] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:56.914957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:56.915131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 34/70] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:39:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:39:56.917221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:57.050440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:57.071285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:57.106434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda7051-4e4f-4fd0-a8a8-ff6250b4e6eb None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:57.115711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1776c6b-e050-4ac0-80a9-ffdaeab53e26 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:57.179246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c352c-1b35-4357-a308-74c63a26d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:57.180745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c352c-1b35-4357-a308-74c63a26d26b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.181094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:57.181352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c352c-1b35-4357-a308-74c63a26d26b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.181799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c352c-1b35-4357-a308-74c63a26d26b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.182558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.183024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.183525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] PATH_INFO: `/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.185916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] RBAC: Authorizing `identity:get_user(user_id=b8adc1b6aec146cdaed3b9ac9a211c0a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:57.188501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:57.191452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 35/71] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:39:56 2022] GET /identity/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:57.192875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:57.209625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c352c-1b35-4357-a308-74c63a26d26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4d0c352c-1b35-4357-a308-74c63a26d26b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:39:57.224267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:57.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:39:57.228947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 37/72] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:39:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:39:57.230076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269f04-b7b0-465f-9d9a-ba2808090b76 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.230076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269f04-b7b0-465f-9d9a-ba2808090b76 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.230267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269f04-b7b0-465f-9d9a-ba2808090b76 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.232918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 38/73] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:57.234430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f269f04-b7b0-465f-9d9a-ba2808090b76 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:57.382604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f269f04-b7b0-465f-9d9a-ba2808090b76 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:57.409240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2758b37-adb2-47c1-b6a6-d72806bfaf47 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:57.431887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:57.432571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.432882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.433178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.451784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:57.474681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:57.475983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.476500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.477018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.478261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:57.478591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:57.481292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:57.485854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:57.505944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 36/74] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:39:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:57.505944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:57.572209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:57.575837 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:57.655163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950735b2-874b-4243-b0cc-1949fc6022e3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:57.697694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366b296-7a6b-41c0-8a20-5765e47c50b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:57.698535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b296-7a6b-41c0-8a20-5765e47c50b0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.698916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b296-7a6b-41c0-8a20-5765e47c50b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.699288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366b296-7a6b-41c0-8a20-5765e47c50b0 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.700269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366b296-7a6b-41c0-8a20-5765e47c50b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:57.700737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366b296-7a6b-41c0-8a20-5765e47c50b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:57.702660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366b296-7a6b-41c0-8a20-5765e47c50b0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:57.714505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 37/75] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:39:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:57.716884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b71ce5-2244-4e59-adeb-dfd422cdb036 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.717382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b71ce5-2244-4e59-adeb-dfd422cdb036 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.717888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b71ce5-2244-4e59-adeb-dfd422cdb036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.740314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-43b71ce5-2244-4e59-adeb-dfd422cdb036 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:57.741809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-43b71ce5-2244-4e59-adeb-dfd422cdb036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:57.795279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 39/76] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:39:57.796900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:57.839742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:57.855624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 38/77] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:57.857262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ab8ad-a377-4513-9b7d-1eb087002535 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:57.857700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ab8ad-a377-4513-9b7d-1eb087002535 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:57.858118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ab8ad-a377-4513-9b7d-1eb087002535 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:57.859787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 39/78] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:39:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:39:57.860739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ab8ad-a377-4513-9b7d-1eb087002535 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:57.881353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:57.984775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b82b-8688-4478-8f69-5b645831c68f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:58.020824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ab8ad-a377-4513-9b7d-1eb087002535 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:58.043523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:58.045065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:58.045566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:58.046033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:58.072059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:58.073876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcde65-9569-4be4-9a8f-79315fe420f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:58.074864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:58.075018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcde65-9569-4be4-9a8f-79315fe420f8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:58.075269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcde65-9569-4be4-9a8f-79315fe420f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:58.075608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcde65-9569-4be4-9a8f-79315fe420f8 None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:58.077257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcde65-9569-4be4-9a8f-79315fe420f8 None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:58.078165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 40/79] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:39:57 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:58.079109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcde65-9569-4be4-9a8f-79315fe420f8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:58.079630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:58.081716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 40/80] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:39:57 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:58.083007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f43d9-fef4-4051-a84a-6d86a9b76a08 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:58.083436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f43d9-fef4-4051-a84a-6d86a9b76a08 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:58.083885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f43d9-fef4-4051-a84a-6d86a9b76a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:58.106863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-140f43d9-fef4-4051-a84a-6d86a9b76a08 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:39:58.108450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-140f43d9-fef4-4051-a84a-6d86a9b76a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:39:58.179129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:58.213751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 41/81] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:39:58.213751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f43d9-fef4-4051-a84a-6d86a9b76a08 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:58.271547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:58.275725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fbc7-5235-45fd-a780-670abdec2612 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:58.341262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:58.342857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:58.343683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:58.343934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:58.346337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:58.349252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:58.352659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 41/82] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:39:58 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:58.354037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:58.388433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f43d9-fef4-4051-a84a-6d86a9b76a08 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:58.447863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:58.448786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:58.449105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:58.449382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:58.450624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:58.452100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:58.453857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 42/83] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:58.454657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:58.524059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ece00d-9120-4179-ba45-1979b1607447 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:58.557940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:58.581701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:58.582776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:58.583276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:58.583726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:58.624342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ca350-1e2f-479a-bc59-d7a5b21b8491 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:58.635224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:58.680327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:58.681047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:58.681335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:58.681622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:58.708769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:58.711896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:58.795091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:58.797909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:58.870533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 42/84] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:39:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:39:58.871928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:58.896252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:58.972013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 43/85] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:39:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:39:58.973880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:58.986724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:59.053923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fafbe-3571-4140-a811-ec56d2c0b8d6 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:59.115310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:59.117490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:59.118016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:59.118564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:59.139123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:39:59.152491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb14fd4-366e-418a-a34e-3a3848478090 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:39:59.154255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:39:59.155274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:39:59.156034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:39:59.156699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 43/86] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:39:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:39:59.157795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:59.230045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:59.231230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:59.231751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:59.232264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:59.233992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=b8adc1b6aec146cdaed3b9ac9a211c0a) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:59.234583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:59.237536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:59.268874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 44/87] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:39:58 2022] GET /identity/v3/role_assignments?user.id=b8adc1b6aec146cdaed3b9ac9a211c0a&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:59.268874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:59.305008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:59.312206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4690b4-3fa4-4c90-9fe5-a9d4233cc904 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:59.358136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:59.372213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:59.373080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:59.373484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:59.373836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:59.375500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:59.377316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:59.419169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 44/88] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:39:59 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:39:59.420724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:59.480488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223ec1-0ace-43e3-8084-faaa7f006ae8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:59.555347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:59.556710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:59.557702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:59.558084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:59.560568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:59.563839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:59.568050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 45/89] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:39:59 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:59.569695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:59.596685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0bb24-8e11-4df7-9f53-a8452d4abb4d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:59.649987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:59.651236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:59.651835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:59.652304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:59.654212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:59.656201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:59.659538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 45/90] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:39:59 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:59.660808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:59.755636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:59.758514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:39:59.787985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702177a-0e65-4f10-b217-ed0b96ce528e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:59.849746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:59.850614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:59.850917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:59.851205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:59.852014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:59.852333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:59.854448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:59.862282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e2415-7f04-4e75-bf01-5d2bf96820b7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:39:59.865153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 46/91] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:39:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:59.866388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:39:59.942017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:39:59.942817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:39:59.943115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:39:59.943456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:39:59.944429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=c2befa26411b4fe5943e212d37990575) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:39:59.944764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:39:59.946637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:39:59.953658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 46/92] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:39:59 2022] GET /identity/v3/role_assignments?user.id=c2befa26411b4fe5943e212d37990575&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:39:59.954819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:00.060281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30286-7d85-414e-a5c5-8fe27a9625b5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:00.097142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a879a51-1073-427e-9a40-d707cfa97b08 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:00.120128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:00.129403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:00.130568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:00.131102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:00.131674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:00.133920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] RBAC: Adding query filter params (role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=c2befa26411b4fe5943e212d37990575) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:00.134720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:00.137851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:00.148103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:00.149056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:00.149353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:00.149641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:00.151064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:00.151717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 47/93] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:39:59 2022] GET /identity/v3/role_assignments?user.id=c2befa26411b4fe5943e212d37990575&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:00.152618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:00.155096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 47/94] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:39:59 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:00.165267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:00.200010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:00.376666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da05f-f408-4430-b289-7f30526383ab None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:00.446367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:00.447703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:00.448507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:00.448507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:00.594604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:00.597635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:00.628741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:00.801880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 48/95] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:00.812119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:01.092280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05673a0c-d8ab-4b4a-8b2b-8ff7b6d45ad4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:01.184394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fe0560-204e-4088-9ebb-e8e76ad84470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:01.187190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe0560-204e-4088-9ebb-e8e76ad84470 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:01.188039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe0560-204e-4088-9ebb-e8e76ad84470 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:01.188900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fe0560-204e-4088-9ebb-e8e76ad84470 None admin] PATH_INFO: `/v3/projects/5e71c8d4b4614065a456c67f81ab2a39` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:01.191622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe0560-204e-4088-9ebb-e8e76ad84470 None admin] RBAC: Authorizing `identity:get_project(project_id=5e71c8d4b4614065a456c67f81ab2a39)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:01.195444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fe0560-204e-4088-9ebb-e8e76ad84470 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:01.199665 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 48/96] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:00 2022] GET /identity/v3/projects/5e71c8d4b4614065a456c67f81ab2a39 => generated 312 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:01.209411 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:01.426895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f954e4-8b3d-4b5a-a339-e6ae7783790f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:01.449601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:01.478572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623b020-00dd-4cfb-bc55-b46a001ef50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:01.479268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623b020-00dd-4cfb-bc55-b46a001ef50e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:01.479553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623b020-00dd-4cfb-bc55-b46a001ef50e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:01.479841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623b020-00dd-4cfb-bc55-b46a001ef50e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:01.480702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623b020-00dd-4cfb-bc55-b46a001ef50e None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=5e71c8d4b4614065a456c67f81ab2a39, user.id=c2befa26411b4fe5943e212d37990575) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:01.481069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623b020-00dd-4cfb-bc55-b46a001ef50e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:01.482601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623b020-00dd-4cfb-bc55-b46a001ef50e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:01.494117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 49/97] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:01 2022] GET /identity/v3/role_assignments?user.id=c2befa26411b4fe5943e212d37990575&scope.project.id=5e71c8d4b4614065a456c67f81ab2a39&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:02.883648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e9a91-4de8-4de5-9948-b70b33a89ee6 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:02.886468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e9a91-4de8-4de5-9948-b70b33a89ee6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:02.887365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e9a91-4de8-4de5-9948-b70b33a89ee6 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:02.891233 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 49/98] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:02.899023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff2d11-bb2b-4da2-9573-ed79e72eb73c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:02.899977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff2d11-bb2b-4da2-9573-ed79e72eb73c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:02.899977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff2d11-bb2b-4da2-9573-ed79e72eb73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:02.937548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4dff2d11-bb2b-4da2-9573-ed79e72eb73c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:02.939629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4dff2d11-bb2b-4da2-9573-ed79e72eb73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:03.097132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 50/99] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:03.139653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e9a91-4de8-4de5-9948-b70b33a89ee6 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:03.285696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:03.419923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e9a91-4de8-4de5-9948-b70b33a89ee6 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:03.511915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:03.513314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:03.513819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:03.514332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:03.544452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:03.565003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:03.566789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:03.568232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 50/100] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:40:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:03.575295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff2d11-bb2b-4da2-9573-ed79e72eb73c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:03.902903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff2d11-bb2b-4da2-9573-ed79e72eb73c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:03.933275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:03.987209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efeae5-5d87-4c97-97a6-609f878c32bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:03.988252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efeae5-5d87-4c97-97a6-609f878c32bb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:03.988683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efeae5-5d87-4c97-97a6-609f878c32bb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:03.989159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efeae5-5d87-4c97-97a6-609f878c32bb None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:03.993805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efeae5-5d87-4c97-97a6-609f878c32bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:03.997843 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efeae5-5d87-4c97-97a6-609f878c32bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:03.999893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efeae5-5d87-4c97-97a6-609f878c32bb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:04.014313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 51/101] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:04.025465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:04.266312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267db724-50b6-404f-89aa-404b1091e819 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.266917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267db724-50b6-404f-89aa-404b1091e819 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.267340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267db724-50b6-404f-89aa-404b1091e819 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.268981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 52/102] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:04.278115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857baed6-9b26-4695-9a1b-8cff98a70de4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.279091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857baed6-9b26-4695-9a1b-8cff98a70de4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.279675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857baed6-9b26-4695-9a1b-8cff98a70de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.329177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e166ea9-070f-4ea1-a79e-e5a9ed25e194 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:04.334476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-857baed6-9b26-4695-9a1b-8cff98a70de4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:04.336695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-857baed6-9b26-4695-9a1b-8cff98a70de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:04.379323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:04.462433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:04.464026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.464477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.464971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] PATH_INFO: `/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.467786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] RBAC: Authorizing `identity:get_user(user_id=b8adc1b6aec146cdaed3b9ac9a211c0a)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:04.470664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:04.474337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 51/103] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:04 2022] GET /identity/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a => generated 247 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:04.484663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:04.614170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 53/104] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:04.659495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79baaa0b-03bf-45e2-a6c8-1b6ddbd25573 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.660723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79baaa0b-03bf-45e2-a6c8-1b6ddbd25573 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.661920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79baaa0b-03bf-45e2-a6c8-1b6ddbd25573 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.673695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 54/105] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:04 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:04.683322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557a79-782c-4e25-b3dc-5ab85bea52f2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.684365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557a79-782c-4e25-b3dc-5ab85bea52f2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.685318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557a79-782c-4e25-b3dc-5ab85bea52f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.717684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db72c6b-f32f-4930-ad0f-02214943c175 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:04.734441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c8557a79-782c-4e25-b3dc-5ab85bea52f2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:04.736581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c8557a79-782c-4e25-b3dc-5ab85bea52f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:04.764987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:04.766494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.766981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.767460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.769692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:04.771558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:04.773926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 52/106] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:04 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:04.775764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:04.843894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:04.910139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 55/107] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:04.911869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8557a79-782c-4e25-b3dc-5ab85bea52f2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:04.931667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3ac7b-ca21-4f30-875d-b14f0d8ad85c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:04.982881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eadb33-dbfb-46ab-9417-1806bba9b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:04.985415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eadb33-dbfb-46ab-9417-1806bba9b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.985571 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:04.986714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eadb33-dbfb-46ab-9417-1806bba9b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.987390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eadb33-dbfb-46ab-9417-1806bba9b2a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.989386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eadb33-dbfb-46ab-9417-1806bba9b2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:04.991206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eadb33-dbfb-46ab-9417-1806bba9b2a4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:04.993487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 53/108] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:04.995120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac1c1-9165-41d5-9d85-6f7761d56391 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:04.995600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac1c1-9165-41d5-9d85-6f7761d56391 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:04.996054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac1c1-9165-41d5-9d85-6f7761d56391 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:04.997952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 54/109] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:04.998965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ac1c1-9165-41d5-9d85-6f7761d56391 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:05.129037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8557a79-782c-4e25-b3dc-5ab85bea52f2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:05.140576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ac1c1-9165-41d5-9d85-6f7761d56391 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:05.164522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:05.193179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:05.194568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:05.194864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:05.195144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:05.195531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea5c02-22d7-481d-8d0f-5b5bc7877be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:05.197038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea5c02-22d7-481d-8d0f-5b5bc7877be8 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:05.197499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea5c02-22d7-481d-8d0f-5b5bc7877be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:05.197961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea5c02-22d7-481d-8d0f-5b5bc7877be8 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/b8adc1b6aec146cdaed3b9ac9a211c0a/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:05.201658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea5c02-22d7-481d-8d0f-5b5bc7877be8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f352c1af1440d09ff65044ae139a10, user_id=b8adc1b6aec146cdaed3b9ac9a211c0a, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:05.204283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea5c02-22d7-481d-8d0f-5b5bc7877be8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:05.211223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:05.222325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:05.223264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:05.224134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:05.226623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 55/110] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:05.227677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:05.228577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 56/111] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:04 2022] PUT /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/b8adc1b6aec146cdaed3b9ac9a211c0a/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:05.230725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad216-64af-4867-acfa-27ce54f8de02 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:05.231257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad216-64af-4867-acfa-27ce54f8de02 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:05.231713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad216-64af-4867-acfa-27ce54f8de02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:05.280472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f02ad216-64af-4867-acfa-27ce54f8de02 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:05.283628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f02ad216-64af-4867-acfa-27ce54f8de02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:05.441094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 57/112] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:05.451417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508913-5da4-4ae1-bb90-7f8c77cc7898 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:05.451920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508913-5da4-4ae1-bb90-7f8c77cc7898 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:05.452339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b508913-5da4-4ae1-bb90-7f8c77cc7898 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:05.454942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 58/113] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:05.456361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b508913-5da4-4ae1-bb90-7f8c77cc7898 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:05.477971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9d24a-c33a-446c-b18e-1bdc0e098d78 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:05.533680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:05.585903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20b8c3-520d-4e24-acd1-a6e7294cc6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:05.588211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b8c3-520d-4e24-acd1-a6e7294cc6f4 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:05.589061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b8c3-520d-4e24-acd1-a6e7294cc6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:05.589788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b8c3-520d-4e24-acd1-a6e7294cc6f4 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:05.592148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20b8c3-520d-4e24-acd1-a6e7294cc6f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:05.595379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20b8c3-520d-4e24-acd1-a6e7294cc6f4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:05.644243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 56/114] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:40:05 2022] POST /identity/v3/projects => generated 302 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:40:05.645489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa3c9d-4ba6-44e2-b91d-3789f6a98c19 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:05.645795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa3c9d-4ba6-44e2-b91d-3789f6a98c19 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:05.646081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa3c9d-4ba6-44e2-b91d-3789f6a98c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:05.669590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-54fa3c9d-4ba6-44e2-b91d-3789f6a98c19 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:05.670912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-54fa3c9d-4ba6-44e2-b91d-3789f6a98c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:05.706482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:05.788690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 57/115] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:05.790332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa3c9d-4ba6-44e2-b91d-3789f6a98c19 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:05.798275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b508913-5da4-4ae1-bb90-7f8c77cc7898 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:05.891445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:05.894122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:05.894860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:05.895539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:05.897811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:05.899625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:05.903180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:05.924828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 59/116] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:05.926098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:06.059651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:06.079068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa3c9d-4ba6-44e2-b91d-3789f6a98c19 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:06.088813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afdd4a-9e33-44b2-abc4-e79062c9c8e2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:06.149624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74945196-bdbb-4676-9ad3-8f00c1932e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:06.150941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74945196-bdbb-4676-9ad3-8f00c1932e58 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.151433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74945196-bdbb-4676-9ad3-8f00c1932e58 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.151914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74945196-bdbb-4676-9ad3-8f00c1932e58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.154711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:06.156205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.156663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.157280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.173503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74945196-bdbb-4676-9ad3-8f00c1932e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:06.178204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-74945196-bdbb-4676-9ad3-8f00c1932e58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:06.190234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:06.192041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:06.193615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 58/117] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:06.195450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:06.204787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:06.205701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:06.206289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3eb8a-843c-456f-a745-9f8a12650dc1 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.206289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3eb8a-843c-456f-a745-9f8a12650dc1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.206289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3eb8a-843c-456f-a745-9f8a12650dc1 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.206467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 60/118] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:06.206467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:06.206641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 59/119] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:06.207510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3eb8a-843c-456f-a745-9f8a12650dc1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:06.250003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:06.424369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3eb8a-843c-456f-a745-9f8a12650dc1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:06.428400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b813ea-e26c-4077-9c79-4a7af5291ca7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:06.494146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:06.495934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.496761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.497729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.499785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:06.500667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84528d5-6505-4e52-a57d-f2028f48e0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:06.500876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:06.503288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84528d5-6505-4e52-a57d-f2028f48e0d9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.504278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:06.504436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84528d5-6505-4e52-a57d-f2028f48e0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.505215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84528d5-6505-4e52-a57d-f2028f48e0d9 None admin] PATH_INFO: `/v3/users/fa41c3c405464e25b9061fa3bcefc473` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.529412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 60/120] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:40:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:06.531513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:06.537305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:06.543959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84528d5-6505-4e52-a57d-f2028f48e0d9 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41c3c405464e25b9061fa3bcefc473)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:06.547100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84528d5-6505-4e52-a57d-f2028f48e0d9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:06.550558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 61/121] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:06 2022] GET /identity/v3/users/fa41c3c405464e25b9061fa3bcefc473 => generated 289 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:06.554097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752850b6-3d45-4a41-902f-2cfe613e7d5a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.554319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752850b6-3d45-4a41-902f-2cfe613e7d5a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.554836 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752850b6-3d45-4a41-902f-2cfe613e7d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.586723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-752850b6-3d45-4a41-902f-2cfe613e7d5a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:06.588959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-752850b6-3d45-4a41-902f-2cfe613e7d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:06.693572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:06.694478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 62/122] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:06.695179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752850b6-3d45-4a41-902f-2cfe613e7d5a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:06.735612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97893a26-ef69-4071-98c8-10962c55c947 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:06.801489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:06.802865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.804033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.804557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.806219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] RBAC: Adding query filter params (name=member) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:06.807247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:06.814714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:06.831406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 61/123] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:06.833097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:06.881148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752850b6-3d45-4a41-902f-2cfe613e7d5a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:06.919144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:06.927604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:06.929497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:06.929969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:06.930424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:06.932696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:06.935298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:06.937972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 63/124] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:06 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:06.939454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:07.059082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059589c8-c883-4708-8f3f-32a9f785ba34 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:07.114240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:07.116066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:07.117457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:07.118377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:07.121136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df484489-fbb3-466d-8310-40232eb2791a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:07.144325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:07.187409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:07.189329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:07.190177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:07.191043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:07.212172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:07.229773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:07.231442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:07.232979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:07.234323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:07.234323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:07.238046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 64/125] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:07.239998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:07.363353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:07.406953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 62/126] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:07.408865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:07.446898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e528768-733f-418b-bde3-9f7818d34c95 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:07.495716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:07.512816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:07.512816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:07.512816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:07.512816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:07.513482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:07.513482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:07.514128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 65/127] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:07 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:07.515038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:07.600066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ece1a-0964-46b3-89cd-8a88882cf198 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:07.657398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:07.658812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:07.659742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:07.660210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:07.664221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:07.666657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:07.670987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 63/128] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:07 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:07.671887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:07.752674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22a4af-1003-43e9-810f-2277375f234a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:07.790761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:07.808424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:07.810673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:07.811202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:07.811607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:07.815586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:07.816131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:07.817941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:07.833733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 66/129] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:07.835614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:07.859056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc93bb-5dd8-46ae-b6e1-81bc0a902f1b None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:07.891708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:07.952384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:07.954113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:07.954986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:07.955544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:07.959137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:07.966271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:07.973245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 64/130] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:07 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:07.973245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:08.017840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5b91c-b254-461f-91c7-d4b0dc52981b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:08.083341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:08.085341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:08.086746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:08.086746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:08.089327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:08.093155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:08.099144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 67/131] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:07 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:08.100958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:08.169335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea26cd-563d-4df5-898f-59a91bcd4e67 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:08.217740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:08.251959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:08.256056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:08.256056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:08.256056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/c2befa26411b4fe5943e212d37990575/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:08.260389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f352c1af1440d09ff65044ae139a10, user_id=c2befa26411b4fe5943e212d37990575, role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:08.265200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:08.292005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 65/132] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:07 2022] PUT /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/c2befa26411b4fe5943e212d37990575/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:08.297341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:08.352192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd02d0-da76-4bce-abba-fac057725604 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:08.387885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:08.470792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:08.472982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:08.473874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:08.474734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:08.478352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:08.482274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:08.487395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 68/133] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:08 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:08.489807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:08.575794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da6a76-46a7-447c-a843-b69e47e7a803 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:08.653437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa2ceb-bff2-4fb5-bb2c-4901a9b3a63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:08.654396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa2ceb-bff2-4fb5-bb2c-4901a9b3a63e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:08.654816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa2ceb-bff2-4fb5-bb2c-4901a9b3a63e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:08.655192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa2ceb-bff2-4fb5-bb2c-4901a9b3a63e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:08.656176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa2ceb-bff2-4fb5-bb2c-4901a9b3a63e None admin] RBAC: Adding query filter params (role.id=ad5c06a55fab4b96836a70829887c664, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=fa41c3c405464e25b9061fa3bcefc473) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:08.656690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa2ceb-bff2-4fb5-bb2c-4901a9b3a63e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:08.659705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa2ceb-bff2-4fb5-bb2c-4901a9b3a63e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:08.671815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 66/134] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:08 2022] GET /identity/v3/role_assignments?user.id=fa41c3c405464e25b9061fa3bcefc473&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=ad5c06a55fab4b96836a70829887c664 => generated 259 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:08.674043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0825df-c3fe-47f5-a3b1-46f0190d5c63 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:08.674579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0825df-c3fe-47f5-a3b1-46f0190d5c63 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:08.675088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0825df-c3fe-47f5-a3b1-46f0190d5c63 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:08.677745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 67/135] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:08.679807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0825df-c3fe-47f5-a3b1-46f0190d5c63 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:08.769687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d088634-657e-4908-9c2e-233c18474172 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:08.873014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:08.913435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c3f24-f90a-46c0-a488-f92b91a239bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:08.915598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c3f24-f90a-46c0-a488-f92b91a239bb None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:08.916117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c3f24-f90a-46c0-a488-f92b91a239bb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:08.916639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c3f24-f90a-46c0-a488-f92b91a239bb None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/c2befa26411b4fe5943e212d37990575/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:08.924243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c3f24-f90a-46c0-a488-f92b91a239bb None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f352c1af1440d09ff65044ae139a10, user_id=c2befa26411b4fe5943e212d37990575, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:08.927297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c3f24-f90a-46c0-a488-f92b91a239bb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:08.941997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0825df-c3fe-47f5-a3b1-46f0190d5c63 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:08.945182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:08.952997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 69/136] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:08 2022] PUT /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/c2befa26411b4fe5943e212d37990575/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:08.954961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9e835-372d-4a36-a816-17e3f45ae0a7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:08.955262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9e835-372d-4a36-a816-17e3f45ae0a7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:08.955552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9e835-372d-4a36-a816-17e3f45ae0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:08.985248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fcf9e835-372d-4a36-a816-17e3f45ae0a7 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:08.987873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fcf9e835-372d-4a36-a816-17e3f45ae0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:09.020183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:09.022760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:09.023655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:09.024325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] PATH_INFO: `/v3/projects/5e71c8d4b4614065a456c67f81ab2a39` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:09.026795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=5e71c8d4b4614065a456c67f81ab2a39)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:09.029635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:09.033315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 68/137] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:08 2022] GET /identity/v3/projects/5e71c8d4b4614065a456c67f81ab2a39 => generated 312 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:09.044411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:09.185306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 70/138] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:09.265253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3cd0c8-3433-4987-8fe5-c7d480a9d4c7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:09.310665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9e835-372d-4a36-a816-17e3f45ae0a7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:09.343588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a9a6b-ed8c-4053-8cb2-0056508defc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:09.346129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a9a6b-ed8c-4053-8cb2-0056508defc9 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:09.347219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a9a6b-ed8c-4053-8cb2-0056508defc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:09.348044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a9a6b-ed8c-4053-8cb2-0056508defc9 None admin] PATH_INFO: `/v3/projects/5e71c8d4b4614065a456c67f81ab2a39/users/c2befa26411b4fe5943e212d37990575/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:09.353962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a9a6b-ed8c-4053-8cb2-0056508defc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e71c8d4b4614065a456c67f81ab2a39, user_id=c2befa26411b4fe5943e212d37990575, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:09.357667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a9a6b-ed8c-4053-8cb2-0056508defc9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:09.396879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 69/139] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:09 2022] PUT /identity/v3/projects/5e71c8d4b4614065a456c67f81ab2a39/users/c2befa26411b4fe5943e212d37990575/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:40:09.460037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e9e70-2f1e-453a-84c2-0351fd79f3b9 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:09.461260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e9e70-2f1e-453a-84c2-0351fd79f3b9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:09.462044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e9e70-2f1e-453a-84c2-0351fd79f3b9 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:09.465565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 70/140] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:09.473597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa7bb0-023f-4fbf-87e5-13eb7ce5e307 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:09.477952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa7bb0-023f-4fbf-87e5-13eb7ce5e307 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:09.478836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa7bb0-023f-4fbf-87e5-13eb7ce5e307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:09.538003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fbfa7bb0-023f-4fbf-87e5-13eb7ce5e307 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:09.540981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fbfa7bb0-023f-4fbf-87e5-13eb7ce5e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:09.583744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9e835-372d-4a36-a816-17e3f45ae0a7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:09.616601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:09.627601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:09.666479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:09.667873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:09.668375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:09.668948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:09.691378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:09.707052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:09.707991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:09.708759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:09.710227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 71/141] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:40:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:09.724361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:09.755438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 71/142] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:09.797587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa7bb0-023f-4fbf-87e5-13eb7ce5e307 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:09.933258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967ac28-3eb5-40d9-9c2f-e6fcd4082466 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:10.048885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:10.050119 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:10.050680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:10.051160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:10.052645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:10.053943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:10.056656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:10.086839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 72/143] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:10.098014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:10.151780 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa7bb0-023f-4fbf-87e5-13eb7ce5e307 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:10.168308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:10.290210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:10.291643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:10.292167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:10.292693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:10.298369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:10.301087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:10.304969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 72/144] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:10.311650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:10.379847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136dca6-0c79-4953-b7bf-2cd087ce2b05 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:10.413997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:10.485268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:10.486884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:10.487387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:10.487854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] PATH_INFO: `/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:10.490404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] RBAC: Authorizing `identity:get_user(user_id=b8adc1b6aec146cdaed3b9ac9a211c0a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:10.493370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:10.497365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 73/145] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:10 2022] GET /identity/v3/users/b8adc1b6aec146cdaed3b9ac9a211c0a => generated 247 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:10.512187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:10.670894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ac0c2-5dfc-4d20-9070-9d92561c1966 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:10.796282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:10.797889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:10.798401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:10.798930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:10.800271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:10.806831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:10.832854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a60edd-073c-485d-b77a-397295aa1011 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:10.841975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:10.869418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 73/146] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:40:10 2022] POST /identity/v3/users => generated 283 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:40:10.925010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:10.926063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:10.926588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:10.927104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:11.074487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:11.077386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:11.293677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 74/147] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 783 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:11.304729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:11.479424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:11.506821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af11b9a-a2dc-4da0-9d30-7da3e26a9c61 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:11.557183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602bb68-4ff9-4fb8-ad64-797cf65f504b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:11.557995 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602bb68-4ff9-4fb8-ad64-797cf65f504b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:11.558288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602bb68-4ff9-4fb8-ad64-797cf65f504b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:11.558576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602bb68-4ff9-4fb8-ad64-797cf65f504b None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:11.559979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602bb68-4ff9-4fb8-ad64-797cf65f504b None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:11.561530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602bb68-4ff9-4fb8-ad64-797cf65f504b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:11.563799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 74/148] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:11 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:11.574790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:11.580365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:11.820607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5a29-2a90-4fe8-8c66-e7a67fe45656 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:11.902894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16d19c-1d96-4577-a587-e558ff250c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:11.904111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16d19c-1d96-4577-a587-e558ff250c21 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:11.904615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16d19c-1d96-4577-a587-e558ff250c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:11.905163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16d19c-1d96-4577-a587-e558ff250c21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:11.906918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16d19c-1d96-4577-a587-e558ff250c21 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=b8adc1b6aec146cdaed3b9ac9a211c0a) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:11.907485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16d19c-1d96-4577-a587-e558ff250c21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:11.910486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16d19c-1d96-4577-a587-e558ff250c21 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:11.924162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 75/149] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:11 2022] GET /identity/v3/role_assignments?user.id=b8adc1b6aec146cdaed3b9ac9a211c0a&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:12.623076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a528-d01d-47d8-ad94-560b2f100577 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:12.623878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a528-d01d-47d8-ad94-560b2f100577 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:12.623878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a528-d01d-47d8-ad94-560b2f100577 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:12.625125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 75/150] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:12.633897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1c837-41d5-4dac-87c9-23e7e0e92210 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:12.634618 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1c837-41d5-4dac-87c9-23e7e0e92210 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:12.634618 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1c837-41d5-4dac-87c9-23e7e0e92210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:12.659336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2bc1c837-41d5-4dac-87c9-23e7e0e92210 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:12.660559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2bc1c837-41d5-4dac-87c9-23e7e0e92210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:12.750337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:12.757459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 76/151] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:12.824505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a528-d01d-47d8-ad94-560b2f100577 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:12.972434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a528-d01d-47d8-ad94-560b2f100577 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:13.018545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:13.019518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:13.019839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:13.020125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:13.034917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:13.043930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:13.045751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:13.047323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 76/152] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:13.055656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1c837-41d5-4dac-87c9-23e7e0e92210 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:13.196166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1c837-41d5-4dac-87c9-23e7e0e92210 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:13.243910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:13.244608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:13.244919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:13.245202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:13.245992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:13.246306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:13.247733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:13.261968 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 77/153] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:13.270727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:13.289816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:13.412617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e0117-0f1d-4528-b6af-7c68770972ce None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:13.462384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70889e0-d6ee-4f2e-ad28-2305b76c172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:13.463750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70889e0-d6ee-4f2e-ad28-2305b76c172d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:13.464220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70889e0-d6ee-4f2e-ad28-2305b76c172d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:13.464680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70889e0-d6ee-4f2e-ad28-2305b76c172d None admin] PATH_INFO: `/v3/users/fa41c3c405464e25b9061fa3bcefc473` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:13.466976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70889e0-d6ee-4f2e-ad28-2305b76c172d None admin] RBAC: Authorizing `identity:get_user(user_id=fa41c3c405464e25b9061fa3bcefc473)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:13.469573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70889e0-d6ee-4f2e-ad28-2305b76c172d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:13.472781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 77/154] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:13 2022] GET /identity/v3/users/fa41c3c405464e25b9061fa3bcefc473 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:13.479159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:13.613278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:13.720629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13043dd3-2308-4ed2-8375-4c399337efc8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:13.801132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383cb02-c053-4b2e-a203-cdfab8bfcfdf None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:13.801864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383cb02-c053-4b2e-a203-cdfab8bfcfdf None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:13.802162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383cb02-c053-4b2e-a203-cdfab8bfcfdf None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:13.816303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:13.818016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:13.818612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:13.819140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:13.820401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 78/155] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:13 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:13.826919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:13.829142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045e66-33c0-4ccd-b854-a2ed3cdcd65d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:13.829567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:13.829739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045e66-33c0-4ccd-b854-a2ed3cdcd65d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:13.830013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045e66-33c0-4ccd-b854-a2ed3cdcd65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:13.833457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 78/156] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:13 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:13.840367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:13.871587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-62045e66-33c0-4ccd-b854-a2ed3cdcd65d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:13.873652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-62045e66-33c0-4ccd-b854-a2ed3cdcd65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:14.112105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 79/157] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:14.130047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bda337-12fa-422d-920a-bbf46524479d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:14.187327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:14.188234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62045e66-33c0-4ccd-b854-a2ed3cdcd65d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:14.218285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:14.219873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:14.220394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:14.220924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/fa41c3c405464e25b9061fa3bcefc473/roles/ad5c06a55fab4b96836a70829887c664` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:14.227458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f352c1af1440d09ff65044ae139a10, user_id=fa41c3c405464e25b9061fa3bcefc473, role_id=ad5c06a55fab4b96836a70829887c664)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:14.229349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:14.230587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:14.265657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 79/158] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:13 2022] PUT /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10/users/fa41c3c405464e25b9061fa3bcefc473/roles/ad5c06a55fab4b96836a70829887c664 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:14.440108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62045e66-33c0-4ccd-b854-a2ed3cdcd65d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:14.525271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0a9b4-ced1-486d-8004-f9fc4a2de0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:14.527325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0a9b4-ced1-486d-8004-f9fc4a2de0f3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:14.528283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0a9b4-ced1-486d-8004-f9fc4a2de0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:14.529317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0a9b4-ced1-486d-8004-f9fc4a2de0f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:14.552516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0a9b4-ced1-486d-8004-f9fc4a2de0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-fdb0a9b4-ced1-486d-8004-f9fc4a2de0f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:14.570284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:14.571962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:14.573503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:14.577335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 80/159] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:14.587434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:14.723864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55be7ad-7f8d-4fa0-b881-036010dab3e3 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:14.724116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55be7ad-7f8d-4fa0-b881-036010dab3e3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:14.724455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55be7ad-7f8d-4fa0-b881-036010dab3e3 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:14.730346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 81/160] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:14.737370 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81952-1f07-492b-884e-4fca6e2b014f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:14.737760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81952-1f07-492b-884e-4fca6e2b014f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:14.737963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81952-1f07-492b-884e-4fca6e2b014f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:14.782890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a5b81952-1f07-492b-884e-4fca6e2b014f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:14.785549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a5b81952-1f07-492b-884e-4fca6e2b014f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:14.832621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b0816-5e94-404d-8e33-ce53ab852aa8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:14.933739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563962a-b4ca-443c-96f7-1697964b3ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:14.934870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563962a-b4ca-443c-96f7-1697964b3ae2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:14.935329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563962a-b4ca-443c-96f7-1697964b3ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:14.935622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563962a-b4ca-443c-96f7-1697964b3ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:14.936522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563962a-b4ca-443c-96f7-1697964b3ae2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:14.936935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563962a-b4ca-443c-96f7-1697964b3ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:14.938375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563962a-b4ca-443c-96f7-1697964b3ae2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:14.956635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 80/161] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:40:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:14.958908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0cca1-d169-481f-814c-db6fad8bddba None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:14.959975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0cca1-d169-481f-814c-db6fad8bddba None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:14.960428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0cca1-d169-481f-814c-db6fad8bddba None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:14.962853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 81/162] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:14.972300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd14e54-00ee-493c-a02f-9a16519469b3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:14.972768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd14e54-00ee-493c-a02f-9a16519469b3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:14.973266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd14e54-00ee-493c-a02f-9a16519469b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:15.021486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:15.031509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2cd14e54-00ee-493c-a02f-9a16519469b3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:15.033614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2cd14e54-00ee-493c-a02f-9a16519469b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:15.034118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 82/163] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:15.035522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b81952-1f07-492b-884e-4fca6e2b014f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:15.200079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:15.243518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 82/164] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:15.245073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd14e54-00ee-493c-a02f-9a16519469b3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:15.370303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b81952-1f07-492b-884e-4fca6e2b014f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:15.514868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:15.517127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:15.517913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:15.518644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:15.521109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:15.532876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:15.533950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd14e54-00ee-493c-a02f-9a16519469b3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:15.536776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 83/165] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:15 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:15.538546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:15.610425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:15.611256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:15.611603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:15.611881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:15.635460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:15.652064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:15.653773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 83/166] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:15.655111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:15.729720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:15.755384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:15.884686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dff27-bfa5-4f4f-8072-b5061c154efc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:15.897650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79038e2c-1ea9-4912-a821-3baef4ad5988 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:15.970352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:15.971073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:15.971350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:15.971625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:16.034078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:16.036425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:16.037284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:16.038578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:16.077801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:16.102336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:16.104053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:16.105546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:16.122572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 84/167] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 585 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:16.123942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:16.210630 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:16.220446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:16.373386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:16.580423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b43fba-443e-4e9f-979f-172076ae59d9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:16.641986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 84/168] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 988 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:16.643495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:16.691223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:16.731128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:16.732292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:16.732798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:16.733332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:16.734834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] RBAC: Adding query filter params (name=member) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:16.735406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:16.746451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:16.775183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 85/169] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 652 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:16.781501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:17.048869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4cffa-0d52-4259-bf11-851e3c0b93cf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:17.164767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc411e-308d-4c60-94ac-eada1492c821 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:17.234061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:17.235165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:17.235641 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:17.236106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:17.237500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:17.238022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:17.241006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:17.273019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 85/170] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 630 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:17.276105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:17.286095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:17.287768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:17.288293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:17.288779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:17.289375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:17.294395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:17.297383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:17.300864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 86/171] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:16 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 525 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:17.306592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:17.540880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:17.551687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0f2e2-0048-4ef9-8062-1ae9e3c2793b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:17.619490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:17.621408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:17.621624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:17.622049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:17.624272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:17.633109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:17.633109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 86/172] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:17 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:17.633109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:17.708538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce0a85-cfec-44f9-9e59-270a8c82820e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:17.794532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:17.841802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc2a86-1b7a-44c6-8af9-e41b120b8dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:17.842882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2a86-1b7a-44c6-8af9-e41b120b8dc7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:17.843232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2a86-1b7a-44c6-8af9-e41b120b8dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:17.843518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc2a86-1b7a-44c6-8af9-e41b120b8dc7 None admin] PATH_INFO: `/v3/users/c2befa26411b4fe5943e212d37990575` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:17.845084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc2a86-1b7a-44c6-8af9-e41b120b8dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=c2befa26411b4fe5943e212d37990575)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:17.846603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc2a86-1b7a-44c6-8af9-e41b120b8dc7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:17.849454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 87/173] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:17 2022] GET /identity/v3/users/c2befa26411b4fe5943e212d37990575 => generated 275 bytes in 544 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:17.851927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e31d9-290d-48ea-bb89-12f74370b990 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:17.852435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e31d9-290d-48ea-bb89-12f74370b990 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:17.857604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e31d9-290d-48ea-bb89-12f74370b990 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:17.859817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 88/174] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:17.860718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6db662-8f14-4ae0-92fe-a1c88b13c53c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:17.861227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e31d9-290d-48ea-bb89-12f74370b990 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:17.916629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:17.919441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:17.919842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:17.920119 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:17.921165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] RBAC: Adding query filter params (role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=c2befa26411b4fe5943e212d37990575) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:17.921490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:17.928538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:17.947481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 87/175] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:17 2022] GET /identity/v3/role_assignments?user.id=c2befa26411b4fe5943e212d37990575&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:17.948434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:18.224016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e31d9-290d-48ea-bb89-12f74370b990 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:18.259882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:18.280641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ea31f-cc39-4a39-82bb-7d4d8f0d67fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:18.281878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ea31f-cc39-4a39-82bb-7d4d8f0d67fa None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:18.282380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ea31f-cc39-4a39-82bb-7d4d8f0d67fa None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:18.282883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ea31f-cc39-4a39-82bb-7d4d8f0d67fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:18.284392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e166150-599d-4120-ad4a-a2afd74a31cc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:18.368455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ad5e6-fadd-405f-bb4c-addf71d6bf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:18.369910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ad5e6-fadd-405f-bb4c-addf71d6bf0b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:18.370433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ad5e6-fadd-405f-bb4c-addf71d6bf0b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:18.370900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ad5e6-fadd-405f-bb4c-addf71d6bf0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:18.377559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:18.428549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ea31f-cc39-4a39-82bb-7d4d8f0d67fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:18.430447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ea31f-cc39-4a39-82bb-7d4d8f0d67fa None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:18.496599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ad5e6-fadd-405f-bb4c-addf71d6bf0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:18.498997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ad5e6-fadd-405f-bb4c-addf71d6bf0b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:18.594990 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 89/176] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:18.598593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46436df-651a-44df-b12c-d4ec27107d28 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:18.598593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46436df-651a-44df-b12c-d4ec27107d28 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:18.598593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46436df-651a-44df-b12c-d4ec27107d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:18.620804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-e46436df-651a-44df-b12c-d4ec27107d28 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:18.622115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e46436df-651a-44df-b12c-d4ec27107d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:18.650868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:18.657187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 88/177] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:18.661448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaa6ab-1f8b-4a9b-a5fb-bd956113ebeb None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:18.661448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaa6ab-1f8b-4a9b-a5fb-bd956113ebeb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:18.661448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaa6ab-1f8b-4a9b-a5fb-bd956113ebeb None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:18.667397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 89/178] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:18.667397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c875f7c-09cd-4ff8-9dde-a82d1b709372 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:18.667397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c875f7c-09cd-4ff8-9dde-a82d1b709372 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:18.667397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c875f7c-09cd-4ff8-9dde-a82d1b709372 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:18.669167 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 90/179] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:18.670624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c875f7c-09cd-4ff8-9dde-a82d1b709372 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:18.749476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:18.751338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 90/180] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:18.756960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46436df-651a-44df-b12c-d4ec27107d28 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:18.897407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c875f7c-09cd-4ff8-9dde-a82d1b709372 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:18.956529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46436df-651a-44df-b12c-d4ec27107d28 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:18.968494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e2c28-0518-4ebb-acc9-76094c146238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:18.970782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e2c28-0518-4ebb-acc9-76094c146238 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:18.971376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e2c28-0518-4ebb-acc9-76094c146238 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:18.971891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e2c28-0518-4ebb-acc9-76094c146238 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:18.975296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e2c28-0518-4ebb-acc9-76094c146238 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:18.978432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e2c28-0518-4ebb-acc9-76094c146238 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:18.985359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 91/181] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:18 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:18.988368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf968-9738-4780-8412-88346e95373c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:18.989552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf968-9738-4780-8412-88346e95373c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:18.990030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdf968-9738-4780-8412-88346e95373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.010434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bbc3c-7232-41b2-8e15-4d300499de1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:19.011353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bbc3c-7232-41b2-8e15-4d300499de1b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:19.011640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bbc3c-7232-41b2-8e15-4d300499de1b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:19.011921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bbc3c-7232-41b2-8e15-4d300499de1b None admin] PATH_INFO: `/v3/projects/5e71c8d4b4614065a456c67f81ab2a39` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.013465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bbc3c-7232-41b2-8e15-4d300499de1b None admin] RBAC: Authorizing `identity:get_project(project_id=5e71c8d4b4614065a456c67f81ab2a39)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:19.014993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bbc3c-7232-41b2-8e15-4d300499de1b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:19.017491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 91/182] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:18 2022] GET /identity/v3/projects/5e71c8d4b4614065a456c67f81ab2a39 => generated 312 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:19.021161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7dbdf968-9738-4780-8412-88346e95373c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:19.023809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7dbdf968-9738-4780-8412-88346e95373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:19.024278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526f96-d242-46b9-a187-e98e5addf76c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:19.024760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526f96-d242-46b9-a187-e98e5addf76c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:19.025244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40526f96-d242-46b9-a187-e98e5addf76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.057045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-40526f96-d242-46b9-a187-e98e5addf76c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:19.059110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-40526f96-d242-46b9-a187-e98e5addf76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:19.094737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:19.161033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 92/183] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:19.163069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdf968-9738-4780-8412-88346e95373c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:19.204913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 92/184] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:19.204913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40526f96-d242-46b9-a187-e98e5addf76c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:19.224477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:19.331945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdf968-9738-4780-8412-88346e95373c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:19.374969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:19.375776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:19.376060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:19.376337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.388774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:19.398178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:19.399273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 93/185] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:19.406839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:19.407791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40526f96-d242-46b9-a187-e98e5addf76c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:19.456099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:19.456966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:19.457262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:19.457543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.458408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=c2befa26411b4fe5943e212d37990575) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:19.458724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:19.460789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:19.476924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 93/186] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:19 2022] GET /identity/v3/role_assignments?user.id=c2befa26411b4fe5943e212d37990575&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:19.476924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:19.505592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:19.559728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:19.580483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353ee1c-487c-4c77-826a-ec2cd00dcb51 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:19.655241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:19.656335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:19.656879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:19.657426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.659194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=5e71c8d4b4614065a456c67f81ab2a39, user.id=c2befa26411b4fe5943e212d37990575) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:19.659793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:19.662818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:19.672001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec707c-d63f-4778-a332-a747301bdf03 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:19.680926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 94/187] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:19 2022] GET /identity/v3/role_assignments?user.id=c2befa26411b4fe5943e212d37990575&scope.project.id=5e71c8d4b4614065a456c67f81ab2a39&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:19.680926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:19.762375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cb3a8-4ada-4d28-85b2-d83d7fac2bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:19.764205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb3a8-4ada-4d28-85b2-d83d7fac2bdb None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:19.764639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb3a8-4ada-4d28-85b2-d83d7fac2bdb None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:19.765079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb3a8-4ada-4d28-85b2-d83d7fac2bdb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.792168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cb3a8-4ada-4d28-85b2-d83d7fac2bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d43cb3a8-4ada-4d28-85b2-d83d7fac2bdb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:19.807384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:19.808361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:19.809298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:19.816441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 94/188] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:19.818810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3bf9b-2f2d-48fb-9f76-9735f0683967 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:19.819274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3bf9b-2f2d-48fb-9f76-9735f0683967 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:19.819589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3bf9b-2f2d-48fb-9f76-9735f0683967 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:19.822565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 95/189] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:19.823947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3bf9b-2f2d-48fb-9f76-9735f0683967 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:19.924665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbe149-726d-4b9c-9f28-f9bb2cae0838 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:19.994061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:20.005069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:20.006001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.006368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.006778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.008559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:20.010256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:20.012120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 95/190] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:20.014471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:20.040187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3bf9b-2f2d-48fb-9f76-9735f0683967 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:20.060537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:20.093415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ef761-4037-4824-abd9-95dd1b0e6abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:20.094122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ef761-4037-4824-abd9-95dd1b0e6abb None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.094411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ef761-4037-4824-abd9-95dd1b0e6abb None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.094764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ef761-4037-4824-abd9-95dd1b0e6abb None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.095591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ef761-4037-4824-abd9-95dd1b0e6abb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:20.095959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ef761-4037-4824-abd9-95dd1b0e6abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:20.097451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ef761-4037-4824-abd9-95dd1b0e6abb None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:20.107868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 96/191] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:20.109258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8acf93-e481-4877-a330-670e69b6cb0a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.109605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8acf93-e481-4877-a330-670e69b6cb0a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.109986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8acf93-e481-4877-a330-670e69b6cb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.132125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5f8acf93-e481-4877-a330-670e69b6cb0a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:20.134089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5f8acf93-e481-4877-a330-670e69b6cb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:20.190237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2976e-f7f8-45be-bee3-effad6f20488 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:20.250427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:20.251121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.251330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.251653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.252523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:20.252935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:20.254533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:20.255571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 97/192] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:20.257137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8acf93-e481-4877-a330-670e69b6cb0a None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:20.266792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 96/193] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:40:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:20.267497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:20.387008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:20.426749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:20.435859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8acf93-e481-4877-a330-670e69b6cb0a None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:20.449539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c62ce-16ab-439a-859b-2792a047e6c3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:20.493446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:20.494751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.495227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.495576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] PATH_INFO: `/v3/users/5898aca04e824f2495005a40af29e9d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.502160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:20.502860 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.503145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.503429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.504176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:20.505750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:20.520203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] RBAC: Authorizing `identity:get_user(user_id=5898aca04e824f2495005a40af29e9d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:20.522769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:20.525264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 97/194] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:20 2022] GET /identity/v3/users/5898aca04e824f2495005a40af29e9d8 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:20.526148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:20.539545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 98/195] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:40:20 2022] POST /identity/v3/users => generated 297 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:40:20.540781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:20.712875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8708c-f9a1-498e-be7a-734384e4070b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:20.742749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add8f57-fc2a-4400-ace5-80a253d7d8dd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:20.762721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:20.764183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.764667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.765117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] PATH_INFO: `/v3/users/5898aca04e824f2495005a40af29e9d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.767753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] RBAC: Authorizing `identity:get_user(user_id=5898aca04e824f2495005a40af29e9d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:20.769821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:20.772872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 98/196] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:20 2022] GET /identity/v3/users/5898aca04e824f2495005a40af29e9d8 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:20.774241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:20.796040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:20.799047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:20.821023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:20.822752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:20.823096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:20.823478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:20.846089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:20.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:20.856269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 99/197] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:20.858669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:20.960437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840914d5-c588-4a10-94b9-4e8f041b65f2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:21.042602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bbe63-fb58-4971-a25c-5ccea51bbbcc None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:21.045867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:21.048168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:21.048669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:21.050600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:21.115334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:21.116439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:21.116937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:21.117403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:21.165808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:21.168647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:21.194914 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:21.251790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:21.282813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:21.284857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:21.323058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 99/198] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:21.323865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:21.461075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 100/199] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:21.462241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:21.480942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873b8be-b7b6-4572-a3d0-23dc5777a975 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:21.502040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:21.538739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:21.538739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:21.538739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:21.540426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:21.541277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:21.541623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:21.543978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:21.558046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 100/200] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:21.558989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:21.630156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:21.664802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071029fa-687c-45fb-8845-cbe6d99b256f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:21.745288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:21.746771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:21.747251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:21.748491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:21.750792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:21.752342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:21.755610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 101/201] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:21 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:21.761699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:21.801487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242435b-7e3e-40c2-a43c-488c705dd4d6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:21.874201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:21.875688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:21.876241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:21.876586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:21.878346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:21.881952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:21.886351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 101/202] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:21 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:21.887578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:21.932892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3372d1-8ddc-495a-afbd-5e54a9f94293 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:21.978639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:21.980017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:21.980528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:21.981095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] PATH_INFO: `/v3/users/fa41c3c405464e25b9061fa3bcefc473` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:21.983363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] RBAC: Authorizing `identity:get_user(user_id=fa41c3c405464e25b9061fa3bcefc473)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:21.985213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:21.985542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:21.988355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 102/203] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:21 2022] GET /identity/v3/users/fa41c3c405464e25b9061fa3bcefc473 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:21.990614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:22.024792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:22.111222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a00c-5580-4ccf-b52e-20a04f0a390b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:22.137013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092298d7-bc64-4216-af0f-7683e04c1514 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:22.192044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:22.192987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:22.193442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:22.193584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:22.193873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:22.194676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:22.194785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=5898aca04e824f2495005a40af29e9d8) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:22.195069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:22.195474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:22.195946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:22.197149 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] RBAC: Adding query filter params (role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=5898aca04e824f2495005a40af29e9d8) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:22.197794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:22.198055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:22.199817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:22.208024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 103/204] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:21 2022] GET /identity/v3/role_assignments?user.id=5898aca04e824f2495005a40af29e9d8&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:22.209666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:22.212976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 102/205] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:21 2022] GET /identity/v3/role_assignments?user.id=5898aca04e824f2495005a40af29e9d8&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:22.383624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:22.410647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b31ad-db83-40e0-90e1-a4b519e813c8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:22.467431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:22.468479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:22.468479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:22.468737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:22.584873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:22.586491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:22.714696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:22.735762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 104/206] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:22.742994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:22.951979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d955f-74c0-44d3-a3d4-1c3ce2ace579 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:22.967807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:23.029891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a3fd7-06b6-4ae3-a07f-182ed519b93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:23.031435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a3fd7-06b6-4ae3-a07f-182ed519b93a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:23.031917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a3fd7-06b6-4ae3-a07f-182ed519b93a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:23.032404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a3fd7-06b6-4ae3-a07f-182ed519b93a None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:23.038383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a3fd7-06b6-4ae3-a07f-182ed519b93a None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:23.041279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a3fd7-06b6-4ae3-a07f-182ed519b93a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:23.045300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 103/207] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:22 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:23.051717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:23.254727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14bf42-a78c-410b-9a12-6b49c529cddd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:23.319239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb952f3-f4e4-495d-b8e1-761a875a1036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:23.320081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb952f3-f4e4-495d-b8e1-761a875a1036 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:23.320370 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb952f3-f4e4-495d-b8e1-761a875a1036 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:23.320649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb952f3-f4e4-495d-b8e1-761a875a1036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:23.321714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb952f3-f4e4-495d-b8e1-761a875a1036 None admin] RBAC: Adding query filter params (role.id=ad5c06a55fab4b96836a70829887c664, scope.project.id=d7f352c1af1440d09ff65044ae139a10, user.id=fa41c3c405464e25b9061fa3bcefc473) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:23.322095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb952f3-f4e4-495d-b8e1-761a875a1036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:23.323818 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb952f3-f4e4-495d-b8e1-761a875a1036 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:23.337491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 105/208] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:23 2022] GET /identity/v3/role_assignments?user.id=fa41c3c405464e25b9061fa3bcefc473&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=ad5c06a55fab4b96836a70829887c664 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:24.346405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ae40-801b-4ac8-a3e2-6a98b937ccf0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:24.346968 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ae40-801b-4ac8-a3e2-6a98b937ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:24.347610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ae40-801b-4ac8-a3e2-6a98b937ccf0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:24.351162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 104/209] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:24.360798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38fbb-9e5a-49e6-a750-288f6e330275 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:24.362416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38fbb-9e5a-49e6-a750-288f6e330275 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:24.363184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a38fbb-9e5a-49e6-a750-288f6e330275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:24.394941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c3a38fbb-9e5a-49e6-a750-288f6e330275 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:24.397606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c3a38fbb-9e5a-49e6-a750-288f6e330275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:24.494030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:24.536197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 106/210] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:24.562198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ae40-801b-4ac8-a3e2-6a98b937ccf0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:24.585207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceb90a-cecf-4cec-914e-1469da3ee52c None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:24.585696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceb90a-cecf-4cec-914e-1469da3ee52c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:24.586367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceb90a-cecf-4cec-914e-1469da3ee52c None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:24.590324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 107/211] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:24.602675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292056-f571-42d8-8dce-90cff26f5f24 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:24.603067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292056-f571-42d8-8dce-90cff26f5f24 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:24.604240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292056-f571-42d8-8dce-90cff26f5f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:24.640765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-14292056-f571-42d8-8dce-90cff26f5f24 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:24.643490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-14292056-f571-42d8-8dce-90cff26f5f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:24.778922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ae40-801b-4ac8-a3e2-6a98b937ccf0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:24.796950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 108/212] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:24.823516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292056-f571-42d8-8dce-90cff26f5f24 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:24.863045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:24.864622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:24.865695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:24.866190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:24.885869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:24.895757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:24.896737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:24.897702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:24.901068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 105/213] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:24.908000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:24.953363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:24.993495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292056-f571-42d8-8dce-90cff26f5f24 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:25.012718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:25.049867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:25.051144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.051864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.052376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.054754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:25.056738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:25.059299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 109/214] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:25.063096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c9430-29d3-4804-893b-ff4a50636768 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:25.066157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:25.114719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:25.116491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.117046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.117549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.118565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:25.119106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:25.120795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:25.131540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 106/215] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:25.137519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:25.248737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479673d-a11b-4b50-89b8-9df783e64997 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:25.299189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:25.307006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:25.307736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.308072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.308362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.309482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:25.311176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:25.312455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4da31-3b5c-4bae-963a-62c5b330559d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:25.344876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 110/216] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:40:25 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:40:25.382858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa5305-0fdd-4fa3-ab63-166a28266309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:25.384459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5305-0fdd-4fa3-ab63-166a28266309 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.385082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5305-0fdd-4fa3-ab63-166a28266309 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.385524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa5305-0fdd-4fa3-ab63-166a28266309 None admin] PATH_INFO: `/v3/users/3814f7bd83f0434593423aa057b8417b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.404537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa5305-0fdd-4fa3-ab63-166a28266309 None admin] RBAC: Authorizing `identity:get_user(user_id=3814f7bd83f0434593423aa057b8417b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:25.406438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa5305-0fdd-4fa3-ab63-166a28266309 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:25.408556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 107/217] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:25 2022] GET /identity/v3/users/3814f7bd83f0434593423aa057b8417b => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:25.417184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:25.497136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:25.649253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7a68a-6834-43ed-b1d4-32a6af94e21b None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:25.710844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71267f81-7576-4a6b-b193-ac309ab78674 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.711081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71267f81-7576-4a6b-b193-ac309ab78674 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.711446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71267f81-7576-4a6b-b193-ac309ab78674 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.715418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 108/218] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:25.722516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:25.723682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.724185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.725332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.728344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e3c9f-0181-4284-9dc5-e0ba27ca101e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.729435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e3c9f-0181-4284-9dc5-e0ba27ca101e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.730213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e3c9f-0181-4284-9dc5-e0ba27ca101e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.767236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-116e3c9f-0181-4284-9dc5-e0ba27ca101e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:25.769345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-116e3c9f-0181-4284-9dc5-e0ba27ca101e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:25.895374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:25.898498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:25.927371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 109/219] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:25.933254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4808e9c-edea-4aeb-8b6d-a10b3f9a96a4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.934588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4808e9c-edea-4aeb-8b6d-a10b3f9a96a4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.936086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4808e9c-edea-4aeb-8b6d-a10b3f9a96a4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.939589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 110/220] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:25.947251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b36a6-057e-49ac-b978-847acd9b5d5a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:25.947683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b36a6-057e-49ac-b978-847acd9b5d5a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:25.948153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b36a6-057e-49ac-b978-847acd9b5d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:25.971869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:25.981037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-297b36a6-057e-49ac-b978-847acd9b5d5a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:25.983301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-297b36a6-057e-49ac-b978-847acd9b5d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:26.126442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 111/221] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:26.127564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b36a6-057e-49ac-b978-847acd9b5d5a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:26.128476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 111/222] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:26.135155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:26.153359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:26.261718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b36a6-057e-49ac-b978-847acd9b5d5a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:26.332314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb625ca-9feb-4994-b7ed-2ebff9619b2d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:26.339873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:26.341322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:26.341898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:26.342386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:26.364381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:26.381609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:26.384050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 112/223] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:40:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:26.386557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:26.413982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:26.415523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:26.416025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:26.416513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:26.419301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:26.421867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:26.425293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 112/224] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:26 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:26.426872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:26.432107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:26.578208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:26.630140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273d89-c4d4-44dd-a1ee-12e77f6b5013 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:26.647137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7dad9-db5d-45d7-b7ff-03e31a06f218 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:26.704662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:26.706218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:26.706841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:26.707184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:26.727498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:26.729011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:26.729703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:26.730611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:26.731818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:26.732594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:26.733400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:26.736061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:26.749603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:26.751355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:26.753734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 113/225] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:26.754943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:26.758486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 113/226] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:26.768615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:26.912585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:26.967125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373e010-08de-4105-9fb3-4404d25c9e99 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:27.007533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe65d5e-1129-4d0a-b44f-d0adbf772b1d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:27.051447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:27.052589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:27.053736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:27.054264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:27.055904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=3814f7bd83f0434593423aa057b8417b) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:27.056484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:27.059782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:27.063527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:27.080896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 114/227] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:26 2022] GET /identity/v3/role_assignments?user.id=3814f7bd83f0434593423aa057b8417b&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:27.080896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:27.087157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:27.089039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:27.089748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:27.090403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] PATH_INFO: `/v3/users/5898aca04e824f2495005a40af29e9d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:27.101144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=5898aca04e824f2495005a40af29e9d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:27.104450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:27.108416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 114/228] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:26 2022] GET /identity/v3/users/5898aca04e824f2495005a40af29e9d8 => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:27.118512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:27.286882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76413f73-f697-42e7-9d06-4ae7a5a7a608 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:27.359265 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:27.360705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:27.361221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:27.361728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:27.363344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:27.363916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:27.366650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:27.384579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 115/229] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:40:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:27.396186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:27.460592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1475ae-a8ea-4553-9196-8a9983b49ae7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:27.519051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:27.534566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:27.546309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:27.548282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:27.549116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:27.549892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:27.553533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:27.556876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:27.561223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 115/230] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:27 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:27.576806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:27.701088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c238a-37bd-40f3-be04-71ee66a066ee None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:27.817685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530856d-cb43-40f1-8076-888bbec5c250 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:27.834987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:27.835919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:27.836200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:27.836470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] PATH_INFO: `/v3/users/5898aca04e824f2495005a40af29e9d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:27.841832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] RBAC: Authorizing `identity:get_user(user_id=5898aca04e824f2495005a40af29e9d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:27.843362 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:27.845246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 116/231] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:27 2022] GET /identity/v3/users/5898aca04e824f2495005a40af29e9d8 => generated 283 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:27.864972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:27.933946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:27.935571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:27.936071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:27.936583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867/users/5898aca04e824f2495005a40af29e9d8/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:27.940760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9cdaf5401b6417a87a138d155210867, user_id=5898aca04e824f2495005a40af29e9d8, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:27.943810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:27.983723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 116/232] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:27 2022] PUT /identity/v3/projects/e9cdaf5401b6417a87a138d155210867/users/5898aca04e824f2495005a40af29e9d8/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:28.123303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91308cc2-c6d8-4d31-a8e9-81142367580e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:28.188359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:28.203867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a0201-a6ac-4e9a-8bc2-c60d8217889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:28.205277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0201-a6ac-4e9a-8bc2-c60d8217889f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:28.205707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0201-a6ac-4e9a-8bc2-c60d8217889f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:28.206114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a0201-a6ac-4e9a-8bc2-c60d8217889f None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:28.208240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a0201-a6ac-4e9a-8bc2-c60d8217889f None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:28.210901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a0201-a6ac-4e9a-8bc2-c60d8217889f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:28.214171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 117/233] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:27 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:28.222865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:28.327461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:28.394724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fd545-86bb-4579-ae6f-f6e02489fbb4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:28.449953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43442b-7f86-4d9a-8784-03882a958df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:28.450888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43442b-7f86-4d9a-8784-03882a958df6 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:28.451174 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43442b-7f86-4d9a-8784-03882a958df6 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:28.451452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43442b-7f86-4d9a-8784-03882a958df6 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867/users/5898aca04e824f2495005a40af29e9d8/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:28.453953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43442b-7f86-4d9a-8784-03882a958df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9cdaf5401b6417a87a138d155210867, user_id=5898aca04e824f2495005a40af29e9d8, role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:28.455461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43442b-7f86-4d9a-8784-03882a958df6 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:28.477953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 117/234] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:28 2022] PUT /identity/v3/projects/e9cdaf5401b6417a87a138d155210867/users/5898aca04e824f2495005a40af29e9d8/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:28.603530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780c4f-2cac-4ce0-860f-65825a72427a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:28.604069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780c4f-2cac-4ce0-860f-65825a72427a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:28.604069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8780c4f-2cac-4ce0-860f-65825a72427a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:28.606090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 118/235] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:28.612266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283ccd4-5521-4182-96c9-aaea270870b1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:28.612587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283ccd4-5521-4182-96c9-aaea270870b1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:28.612937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283ccd4-5521-4182-96c9-aaea270870b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:28.632436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2283ccd4-5521-4182-96c9-aaea270870b1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:28.633758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2283ccd4-5521-4182-96c9-aaea270870b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:28.723246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 118/236] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:28.770454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8780c4f-2cac-4ce0-860f-65825a72427a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:28.954991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8780c4f-2cac-4ce0-860f-65825a72427a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:29.004828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99c5bf-6548-4652-affb-4e59d4c45725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:29.005604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99c5bf-6548-4652-affb-4e59d4c45725 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:29.006491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99c5bf-6548-4652-affb-4e59d4c45725 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:29.006730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99c5bf-6548-4652-affb-4e59d4c45725 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:29.031848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99c5bf-6548-4652-affb-4e59d4c45725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ab99c5bf-6548-4652-affb-4e59d4c45725 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:29.040147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:29.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:29.043516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:29.048896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 119/237] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:29.054680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283ccd4-5521-4182-96c9-aaea270870b1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:29.071652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef11ad-758a-4fca-af14-32bcfe5fcba6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:29.072059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef11ad-758a-4fca-af14-32bcfe5fcba6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:29.072355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef11ad-758a-4fca-af14-32bcfe5fcba6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:29.074532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 120/238] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:29.083076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc56b5b-b62b-41b1-b07c-adb63e31d0a2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:29.083477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc56b5b-b62b-41b1-b07c-adb63e31d0a2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:29.083771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc56b5b-b62b-41b1-b07c-adb63e31d0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:29.121447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:29.136150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-cbc56b5b-b62b-41b1-b07c-adb63e31d0a2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:29.148890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cbc56b5b-b62b-41b1-b07c-adb63e31d0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:29.188676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283ccd4-5521-4182-96c9-aaea270870b1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:29.236900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:29.236900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:29.236900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:29.236900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:29.237542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:29.237542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:29.237542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:29.253090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 119/239] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:29.254706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:29.326853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:29.363174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 121/240] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 14:40:29.405207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855bea-7328-4a00-afac-fd02594ca94a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:29.435358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:29.444808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc56b5b-b62b-41b1-b07c-adb63e31d0a2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:29.482987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:29.484709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:29.485245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:29.485760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] PATH_INFO: `/v3/users/9a516dc1cad045019d0d57a0efa826ad` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:29.513217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] RBAC: Authorizing `identity:get_user(user_id=9a516dc1cad045019d0d57a0efa826ad)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:29.515705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:29.525031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 120/241] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:29 2022] GET /identity/v3/users/9a516dc1cad045019d0d57a0efa826ad => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:29.528339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:29.745283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c4374-3b55-4ceb-8fe0-1634e8bb02d9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:29.793734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc56b5b-b62b-41b1-b07c-adb63e31d0a2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:29.820142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:29.821694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:29.822980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:29.823878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:29.912267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:29.926495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:29.927572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:29.927872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:29.928145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:29.929826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:29.931540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:29.942091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 122/242] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:29.951017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:29.963506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:29.965461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:30.095486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 121/243] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:40:30.097289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:30.102651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:30.212389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe56c0a-3b4d-4e11-8ae8-d4b02ec444b0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:30.245441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:30.289571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a04807-a25f-442a-a2d7-6494543eb19d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:30.326941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd53d0b-08c4-477e-b265-55a27dfcaa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:30.328160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53d0b-08c4-477e-b265-55a27dfcaa16 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:30.328653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53d0b-08c4-477e-b265-55a27dfcaa16 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:30.329192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53d0b-08c4-477e-b265-55a27dfcaa16 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:30.330969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd53d0b-08c4-477e-b265-55a27dfcaa16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:30.340891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:30.340891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:30.341273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd53d0b-08c4-477e-b265-55a27dfcaa16 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:30.341447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:30.341447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:30.341447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:30.342008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:30.349184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 122/244] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:30 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:30.351849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:30.397638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 123/245] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:40:29 2022] POST /identity/v3/users => generated 293 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:40:30.486449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95701d16-02fb-410e-bbe9-bd1ab7cd6559 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:30.551549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97f9d8-9050-4fb7-b935-583f08ea4298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:30.553355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97f9d8-9050-4fb7-b935-583f08ea4298 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:30.553887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97f9d8-9050-4fb7-b935-583f08ea4298 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:30.554376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97f9d8-9050-4fb7-b935-583f08ea4298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:30.555858 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97f9d8-9050-4fb7-b935-583f08ea4298 None admin] RBAC: Adding query filter params (role.id=ad5c06a55fab4b96836a70829887c664, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=9a516dc1cad045019d0d57a0efa826ad) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:30.556198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97f9d8-9050-4fb7-b935-583f08ea4298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:30.557822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97f9d8-9050-4fb7-b935-583f08ea4298 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:30.565040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 123/246] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:30 2022] GET /identity/v3/role_assignments?user.id=9a516dc1cad045019d0d57a0efa826ad&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=ad5c06a55fab4b96836a70829887c664 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:30.664124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1093c-cc23-447c-a1f1-e9deb43479f6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:30.665181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1093c-cc23-447c-a1f1-e9deb43479f6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:30.666023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1093c-cc23-447c-a1f1-e9deb43479f6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:30.669268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 124/247] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:30.675221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2400-03e4-4bc0-bb72-cd1490f1fa17 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:30.675508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2400-03e4-4bc0-bb72-cd1490f1fa17 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:30.675811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e2400-03e4-4bc0-bb72-cd1490f1fa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:30.686268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:30.713401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5d2e2400-03e4-4bc0-bb72-cd1490f1fa17 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:30.714670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5d2e2400-03e4-4bc0-bb72-cd1490f1fa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:30.812781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 124/248] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:30.841098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1093c-cc23-447c-a1f1-e9deb43479f6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:30.996143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1093c-cc23-447c-a1f1-e9deb43479f6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:31.012584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:31.048602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:31.049785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:31.050323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:31.051024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:31.065203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:31.074610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:31.075644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:31.076529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:31.079165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 125/249] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:40:31.089820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2400-03e4-4bc0-bb72-cd1490f1fa17 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:31.243919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e2400-03e4-4bc0-bb72-cd1490f1fa17 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:31.301552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:31.310155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:31.311855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:31.312770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:31.313498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:31.315511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] RBAC: Adding query filter params (name=member) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:31.316503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:31.320530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:31.339708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 125/250] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:31.350094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:31.587709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169c0de-60ab-4947-80c9-a894b49e9e73 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:31.679371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4570406-b811-4590-a788-d351e40903ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:31.681536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4570406-b811-4590-a788-d351e40903ea None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:31.682303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4570406-b811-4590-a788-d351e40903ea None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:31.683018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4570406-b811-4590-a788-d351e40903ea None admin] PATH_INFO: `/v3/users/3814f7bd83f0434593423aa057b8417b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:31.690557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4570406-b811-4590-a788-d351e40903ea None admin] RBAC: Authorizing `identity:get_user(user_id=3814f7bd83f0434593423aa057b8417b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:31.693880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4570406-b811-4590-a788-d351e40903ea None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:31.698042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 126/251] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:31 2022] GET /identity/v3/users/3814f7bd83f0434593423aa057b8417b => generated 297 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:31.707001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:31.922131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6f5da-abf2-43a1-afb6-ac5c7881cace None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:31.983852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd2cf9-480e-4df4-9454-31cd7efdc9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:31.985402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd2cf9-480e-4df4-9454-31cd7efdc9f9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:31.985865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd2cf9-480e-4df4-9454-31cd7efdc9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:31.986318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd2cf9-480e-4df4-9454-31cd7efdc9f9 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:31.988485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd2cf9-480e-4df4-9454-31cd7efdc9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:31.991612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd2cf9-480e-4df4-9454-31cd7efdc9f9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:31.994375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 126/252] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:31 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:32.019846 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4570406-b811-4590-a788-d351e40903ea None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:32.106257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:32.250409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4570406-b811-4590-a788-d351e40903ea None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:32.327328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d3783-a876-4776-b5e9-ad6ea1abfff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:32.328938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d3783-a876-4776-b5e9-ad6ea1abfff7 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:32.329403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d3783-a876-4776-b5e9-ad6ea1abfff7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:32.329847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d3783-a876-4776-b5e9-ad6ea1abfff7 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867/users/3814f7bd83f0434593423aa057b8417b/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:32.336517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d3783-a876-4776-b5e9-ad6ea1abfff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9cdaf5401b6417a87a138d155210867, user_id=3814f7bd83f0434593423aa057b8417b, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:32.339971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d3783-a876-4776-b5e9-ad6ea1abfff7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:32.367944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 127/253] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:32 2022] PUT /identity/v3/projects/e9cdaf5401b6417a87a138d155210867/users/3814f7bd83f0434593423aa057b8417b/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:32.583614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831389d-72b0-4716-aa71-7d56fa0a6837 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:32.585111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831389d-72b0-4716-aa71-7d56fa0a6837 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:32.585794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831389d-72b0-4716-aa71-7d56fa0a6837 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:32.589607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 127/254] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:32.600374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2d7df-4cae-4d96-a5f1-b8f9b222ee3e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:32.601341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2d7df-4cae-4d96-a5f1-b8f9b222ee3e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:32.602128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2d7df-4cae-4d96-a5f1-b8f9b222ee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:32.640307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1ca2d7df-4cae-4d96-a5f1-b8f9b222ee3e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:32.643131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1ca2d7df-4cae-4d96-a5f1-b8f9b222ee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:32.803378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 128/255] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:32.856707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453183f-ba79-4334-a324-4ea95b4c5200 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:32.857758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453183f-ba79-4334-a324-4ea95b4c5200 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:32.858285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453183f-ba79-4334-a324-4ea95b4c5200 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:32.860047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 128/256] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:32.861619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2d7df-4cae-4d96-a5f1-b8f9b222ee3e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:32.876617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0ca3-6b38-4c98-b7eb-39f4117c8610 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:32.877312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0ca3-6b38-4c98-b7eb-39f4117c8610 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:32.877824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0ca3-6b38-4c98-b7eb-39f4117c8610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:32.904564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8efa0ca3-6b38-4c98-b7eb-39f4117c8610 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:32.908577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8efa0ca3-6b38-4c98-b7eb-39f4117c8610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:32.947037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:33.013422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:33.051933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 129/257] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:33.183394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa0ca3-6b38-4c98-b7eb-39f4117c8610 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:33.225779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2d7df-4cae-4d96-a5f1-b8f9b222ee3e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:33.361909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:33.363177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:33.363696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:33.364423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:33.405900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:33.438347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:33.441303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:33.443387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 129/258] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 579 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:33.451999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:33.559184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa0ca3-6b38-4c98-b7eb-39f4117c8610 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:33.675347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:33.676377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:33.676674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:33.677029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:33.695267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:33.707671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:33.708643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:33.709526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 130/259] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 527 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:33.717239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:33.749440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:33.808063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfdccb-b829-47d7-bf63-044cfc7babf8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:33.909020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:33.954412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:33.955480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:33.955977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:33.956457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:33.958038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:33.959456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:33.961558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:33.984345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746df9e4-3705-4bc5-a90c-bd98036ba3f3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:33.996458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 130/260] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:40:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 542 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:34.022878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:34.036785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0682820-2786-46e0-be87-24e408400aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:34.038774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0682820-2786-46e0-be87-24e408400aed None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:34.039486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0682820-2786-46e0-be87-24e408400aed None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:34.040324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0682820-2786-46e0-be87-24e408400aed None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:34.041457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0682820-2786-46e0-be87-24e408400aed None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:34.041945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0682820-2786-46e0-be87-24e408400aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:34.043971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0682820-2786-46e0-be87-24e408400aed None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:34.057191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 131/261] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:34.067068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0682820-2786-46e0-be87-24e408400aed None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:34.363591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0682820-2786-46e0-be87-24e408400aed None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:34.393286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb752c-44fc-4d2e-9962-710c1fe0d74c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:34.437348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:34.450963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:34.452539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:34.453083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:34.453579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] PATH_INFO: `/v3/users/5898aca04e824f2495005a40af29e9d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:34.456078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] RBAC: Authorizing `identity:get_user(user_id=5898aca04e824f2495005a40af29e9d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:34.459214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:34.462619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 132/262] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:34 2022] GET /identity/v3/users/5898aca04e824f2495005a40af29e9d8 => generated 283 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:34.482843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:34.495107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:34.496339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:34.496625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:34.496948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] PATH_INFO: `/v3/users/5898aca04e824f2495005a40af29e9d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:34.498654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] RBAC: Authorizing `identity:get_user(user_id=5898aca04e824f2495005a40af29e9d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:34.501222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:34.504909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 131/263] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:34 2022] GET /identity/v3/users/5898aca04e824f2495005a40af29e9d8 => generated 283 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:34.538396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:34.584647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:34.717568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778b952-0cf1-46e9-b915-c07b8a088c5f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:34.789511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:34.790968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:34.791712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:34.792421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:34.868693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89ff8e-214d-474e-9386-6587ce90ab95 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:34.935436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:35.002198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe605de9-96ef-4d9e-8e96-588475ce8232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:35.003111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605de9-96ef-4d9e-8e96-588475ce8232 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:35.003554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605de9-96ef-4d9e-8e96-588475ce8232 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:35.003978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe605de9-96ef-4d9e-8e96-588475ce8232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:35.018224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:35.020750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:35.238626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe605de9-96ef-4d9e-8e96-588475ce8232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:35.242879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe605de9-96ef-4d9e-8e96-588475ce8232 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:35.276558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 133/264] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:40:35.282962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:35.289500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:35.678964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 132/265] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1141 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:35.681997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9013b-64b2-45c3-9b7f-ca8049bc68ea None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:35.682610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9013b-64b2-45c3-9b7f-ca8049bc68ea None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:35.683123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9013b-64b2-45c3-9b7f-ca8049bc68ea None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:35.686031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 133/266] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:35.694639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9013b-64b2-45c3-9b7f-ca8049bc68ea None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:35.727926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:35.854095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5777a-1043-48fd-9559-d0233fc76ab3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:35.992947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bbe929-b7ed-49d3-8756-33275b6ebfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:35.996423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbe929-b7ed-49d3-8756-33275b6ebfe9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:35.999327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbe929-b7ed-49d3-8756-33275b6ebfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.000102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bbe929-b7ed-49d3-8756-33275b6ebfe9 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.004470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbe929-b7ed-49d3-8756-33275b6ebfe9 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:36.008042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bbe929-b7ed-49d3-8756-33275b6ebfe9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:36.023611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 134/267] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:35 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 742 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:36.026279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab0817-7eda-43a2-b22f-693e5fc3d633 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.028691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab0817-7eda-43a2-b22f-693e5fc3d633 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.031462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab0817-7eda-43a2-b22f-693e5fc3d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.042711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9013b-64b2-45c3-9b7f-ca8049bc68ea None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:36.084813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-04ab0817-7eda-43a2-b22f-693e5fc3d633 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:36.086464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-04ab0817-7eda-43a2-b22f-693e5fc3d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:36.096476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:36.141754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5515a2e-e8ca-4278-a948-605802ab822b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:36.142912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5515a2e-e8ca-4278-a948-605802ab822b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.143241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5515a2e-e8ca-4278-a948-605802ab822b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.143619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5515a2e-e8ca-4278-a948-605802ab822b None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.147422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5515a2e-e8ca-4278-a948-605802ab822b None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:36.151794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5515a2e-e8ca-4278-a948-605802ab822b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:36.157175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 134/268] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:35 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:36.161477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c16f1-e79a-4ac2-bd25-ed60f3c8e4c1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.167305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c16f1-e79a-4ac2-bd25-ed60f3c8e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.169654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c16f1-e79a-4ac2-bd25-ed60f3c8e4c1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.173385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 135/269] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:36 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:36.175583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c16f1-e79a-4ac2-bd25-ed60f3c8e4c1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:36.275430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 135/270] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:36.282713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab0817-7eda-43a2-b22f-693e5fc3d633 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:36.382428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c16f1-e79a-4ac2-bd25-ed60f3c8e4c1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:36.444027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412096de-714d-4e36-8d06-9c0ed1ecab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:36.445257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412096de-714d-4e36-8d06-9c0ed1ecab5e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.445675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412096de-714d-4e36-8d06-9c0ed1ecab5e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.446276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412096de-714d-4e36-8d06-9c0ed1ecab5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.447087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412096de-714d-4e36-8d06-9c0ed1ecab5e None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=5898aca04e824f2495005a40af29e9d8) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:36.447507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412096de-714d-4e36-8d06-9c0ed1ecab5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:36.449654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412096de-714d-4e36-8d06-9c0ed1ecab5e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:36.457952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 136/271] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:36 2022] GET /identity/v3/role_assignments?user.id=5898aca04e824f2495005a40af29e9d8&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:36.459840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb8ac8-fec1-4000-8e1f-aed7e8567385 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.460440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb8ac8-fec1-4000-8e1f-aed7e8567385 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.460786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb8ac8-fec1-4000-8e1f-aed7e8567385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.485737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-38fb8ac8-fec1-4000-8e1f-aed7e8567385 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:36.487525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-38fb8ac8-fec1-4000-8e1f-aed7e8567385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:36.504648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:36.535988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab0817-7eda-43a2-b22f-693e5fc3d633 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:36.601440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae48872-54f3-47cd-8d44-9bc535b40ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:36.602982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48872-54f3-47cd-8d44-9bc535b40ea9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.603753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:36.604051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48872-54f3-47cd-8d44-9bc535b40ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.604775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48872-54f3-47cd-8d44-9bc535b40ea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.606682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae48872-54f3-47cd-8d44-9bc535b40ea9 None admin] RBAC: Adding query filter params (role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=5898aca04e824f2495005a40af29e9d8) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:36.607471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae48872-54f3-47cd-8d44-9bc535b40ea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:36.610571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae48872-54f3-47cd-8d44-9bc535b40ea9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:36.629359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 136/272] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:36 2022] GET /identity/v3/role_assignments?user.id=5898aca04e824f2495005a40af29e9d8&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 608 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:36.631373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6777-d624-4c34-b6e3-c88b9cb9c673 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.631864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6777-d624-4c34-b6e3-c88b9cb9c673 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.632332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6777-d624-4c34-b6e3-c88b9cb9c673 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.640891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 137/273] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:36.640891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab6777-d624-4c34-b6e3-c88b9cb9c673 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:36.684218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 137/274] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:36.687024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96a02-3e04-4f87-b350-61baf5b077f5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:36.687549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96a02-3e04-4f87-b350-61baf5b077f5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:36.688057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d96a02-3e04-4f87-b350-61baf5b077f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:36.750936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-82d96a02-3e04-4f87-b350-61baf5b077f5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:36.753875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-82d96a02-3e04-4f87-b350-61baf5b077f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:36.926018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab6777-d624-4c34-b6e3-c88b9cb9c673 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:36.969608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 138/275] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:36.971599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d96a02-3e04-4f87-b350-61baf5b077f5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:37.002920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:37.004427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.004948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.005438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.019406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:37.034968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:37.036629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:37.039060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:37.043582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 138/276] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:37.045530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:37.054022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:37.212712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d96a02-3e04-4f87-b350-61baf5b077f5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:37.229331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:37.254196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1996929-7bc7-446d-9cd9-49b8645693ec None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:37.274218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:37.275511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.275991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.276499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.300346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:37.312792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:37.313645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.313991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.314413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.315804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:37.317011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:37.326204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:37.328754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:37.330100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:37.336788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 139/277] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:37.337284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 139/278] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:37.337440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:37.337589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:37.478187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:37.540705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8f73a-4147-493d-a138-1b880892008d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:37.563787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231c431-bc61-49fd-a5d8-1351827beba1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:37.607845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:37.608960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.609434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.610034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.611257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:37.611761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:37.614522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:37.627809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 140/279] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:37.629745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:37.639517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082795d0-dcbc-47e7-8a80-799493d35373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:37.642496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082795d0-dcbc-47e7-8a80-799493d35373 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.642496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082795d0-dcbc-47e7-8a80-799493d35373 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.643186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082795d0-dcbc-47e7-8a80-799493d35373 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.644314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082795d0-dcbc-47e7-8a80-799493d35373 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:37.647665 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082795d0-dcbc-47e7-8a80-799493d35373 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:37.688783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 140/280] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:40:37 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:40:37.691763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfaea3-48db-4b7e-9e14-0f1f9f2f56c1 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.692185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfaea3-48db-4b7e-9e14-0f1f9f2f56c1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.692579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfaea3-48db-4b7e-9e14-0f1f9f2f56c1 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.694872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:37.697753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 141/281] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:37.697753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfaea3-48db-4b7e-9e14-0f1f9f2f56c1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:37.849093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534356-f331-4f66-8fc4-1be3c0132670 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:37.909086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfaea3-48db-4b7e-9e14-0f1f9f2f56c1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:37.922393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275a4a1-a49c-49c2-8e7c-f3de2c41c0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:37.923442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275a4a1-a49c-49c2-8e7c-f3de2c41c0e2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.923888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275a4a1-a49c-49c2-8e7c-f3de2c41c0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.924330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275a4a1-a49c-49c2-8e7c-f3de2c41c0e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.925958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275a4a1-a49c-49c2-8e7c-f3de2c41c0e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:37.926486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275a4a1-a49c-49c2-8e7c-f3de2c41c0e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:37.928770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275a4a1-a49c-49c2-8e7c-f3de2c41c0e2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:37.946348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 141/282] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:37.948318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad35774-e524-4515-9ab0-167357b39c3b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:37.948635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad35774-e524-4515-9ab0-167357b39c3b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:37.948947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad35774-e524-4515-9ab0-167357b39c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:37.975203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5ad35774-e524-4515-9ab0-167357b39c3b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:37.976560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5ad35774-e524-4515-9ab0-167357b39c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:37.980432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:38.006292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:38.007825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:38.008286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:38.009373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] PATH_INFO: `/v3/users/db44c711e4814b1c9c62eb7c2cb6b812` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:38.038837 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] RBAC: Authorizing `identity:get_user(user_id=db44c711e4814b1c9c62eb7c2cb6b812)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:38.040742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:38.043766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 142/283] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:37 2022] GET /identity/v3/users/db44c711e4814b1c9c62eb7c2cb6b812 => generated 293 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:38.045968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:38.129327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 142/284] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:38.130587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad35774-e524-4515-9ab0-167357b39c3b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:38.174732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:38.298841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f3a98-551b-4a7b-845a-4cf82f36ae95 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:38.335177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad35774-e524-4515-9ab0-167357b39c3b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:38.391396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:38.393315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:38.393832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:38.394327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] PATH_INFO: `/v3/users/9a516dc1cad045019d0d57a0efa826ad` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:38.397074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] RBAC: Authorizing `identity:get_user(user_id=9a516dc1cad045019d0d57a0efa826ad)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:38.400190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:38.404429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 143/285] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:38 2022] GET /identity/v3/users/9a516dc1cad045019d0d57a0efa826ad => generated 297 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:38.405505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:38.406479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:38.406723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:38.407109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:38.408108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, user.id=db44c711e4814b1c9c62eb7c2cb6b812, scope.system=all) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:38.408495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:38.410967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:38.414433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:38.419015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 143/286] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Jun 16 14:40:38 2022] GET /identity/v3/role_assignments?user.id=db44c711e4814b1c9c62eb7c2cb6b812&scope.system=all&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:38.420698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:38.560903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:38.594382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:38.639321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53d54e-0e9f-4648-bc36-f494944bc65c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:38.729158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaaba8-1941-4d17-98c9-3e112ca45e9f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:38.775070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:38.779062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:38.780241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:38.783029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:38.790441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:38.793828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:38.796209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 144/287] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:38 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:38.815677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:38.866904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:38.868428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:38.869286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:38.870862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:38.915825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:38.932608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:38.934140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:38.936288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 144/288] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 530 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:38.942312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:39.052034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed56742-fe42-4c21-983f-b21763dfa519 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:39.131907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:39.134568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:39.135474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:39.136277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867/users/9a516dc1cad045019d0d57a0efa826ad/roles/ad5c06a55fab4b96836a70829887c664` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:39.147797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9cdaf5401b6417a87a138d155210867, user_id=9a516dc1cad045019d0d57a0efa826ad, role_id=ad5c06a55fab4b96836a70829887c664)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:39.151145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:39.190485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 145/289] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:40:38 2022] PUT /identity/v3/projects/e9cdaf5401b6417a87a138d155210867/users/9a516dc1cad045019d0d57a0efa826ad/roles/ad5c06a55fab4b96836a70829887c664 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:39.226476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d40b8-eba0-4fe2-8dd0-b0d28cf7d1d0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:39.267953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:39.321552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:39.322556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:39.323000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:39.323438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:39.324758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] RBAC: Adding query filter params (name=member) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:39.325312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:39.327888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:39.346101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 145/290] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:39.355049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:39.418348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:39.671768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da81e7-1327-45c0-bec0-7aea2a304896 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:39.755321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:39.757500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:39.758000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:39.758488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] PATH_INFO: `/v3/users/3814f7bd83f0434593423aa057b8417b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:39.760883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] RBAC: Authorizing `identity:get_user(user_id=3814f7bd83f0434593423aa057b8417b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:39.763624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:39.767624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 146/291] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:39 2022] GET /identity/v3/users/3814f7bd83f0434593423aa057b8417b => generated 297 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:39.775152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:40.064057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a6945-cd8e-4915-bdcc-fa5a2858c632 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:40.163446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:40.164477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:40.164935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:40.165426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:40.279485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:40.340376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:40.347793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:40.600317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 146/292] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 826 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:40:40.613136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:40.841676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:40.900909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423325a1-0454-4c97-8927-83cfdc8a8c83 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:40.994116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e5337-ff8e-4a1a-8554-9fadd2f322ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:40.995770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e5337-ff8e-4a1a-8554-9fadd2f322ad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:40.996252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e5337-ff8e-4a1a-8554-9fadd2f322ad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:40.996762 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e5337-ff8e-4a1a-8554-9fadd2f322ad None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:40.999263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e5337-ff8e-4a1a-8554-9fadd2f322ad None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:41.002159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e5337-ff8e-4a1a-8554-9fadd2f322ad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:41.006149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 147/293] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:40 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:41.012281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:41.193558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:41.250131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44386c3-d555-4247-b7ff-b9d3d6343898 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:41.332516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c848b32-7859-4670-a1d9-7163e3533b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:41.333585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848b32-7859-4670-a1d9-7163e3533b04 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:41.333585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848b32-7859-4670-a1d9-7163e3533b04 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:41.333867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848b32-7859-4670-a1d9-7163e3533b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:41.334764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848b32-7859-4670-a1d9-7163e3533b04 None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=3814f7bd83f0434593423aa057b8417b) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:41.335089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848b32-7859-4670-a1d9-7163e3533b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:41.336654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c848b32-7859-4670-a1d9-7163e3533b04 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:41.352492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 147/294] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:41 2022] GET /identity/v3/role_assignments?user.id=3814f7bd83f0434593423aa057b8417b&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:41.950823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f22bc-3d04-431d-ba10-b752f785552c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:41.951604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f22bc-3d04-431d-ba10-b752f785552c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:41.951604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f22bc-3d04-431d-ba10-b752f785552c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:41.953059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 148/295] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:41.963543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae82d-92c0-42b7-914e-4aca19f8c34b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:41.964514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae82d-92c0-42b7-914e-4aca19f8c34b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:41.965363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afae82d-92c0-42b7-914e-4aca19f8c34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:41.996446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5afae82d-92c0-42b7-914e-4aca19f8c34b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:41.999245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5afae82d-92c0-42b7-914e-4aca19f8c34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:42.167414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 148/296] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:42.213552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f22bc-3d04-431d-ba10-b752f785552c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:42.377742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264612a4-3a99-4485-a179-d4114de2df5d None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:42.379333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264612a4-3a99-4485-a179-d4114de2df5d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:42.380135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264612a4-3a99-4485-a179-d4114de2df5d None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:42.386238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 149/297] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:42.395781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fc443-aa7e-4701-97f0-309efece70e4 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:42.397536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fc443-aa7e-4701-97f0-309efece70e4 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:42.398755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fc443-aa7e-4701-97f0-309efece70e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:42.433810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f22bc-3d04-431d-ba10-b752f785552c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:42.441378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-705fc443-aa7e-4701-97f0-309efece70e4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:42.444643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-705fc443-aa7e-4701-97f0-309efece70e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:42.500425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:42.501300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:42.501582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:42.501914 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:42.522333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:42.530951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:42.532701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:42.534314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 149/298] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:42.548781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:42.581247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:42.612313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:42.632885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 150/299] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:42.666287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fc443-aa7e-4701-97f0-309efece70e4 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:42.813823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64b56-96d4-4aad-a781-f26c81dbd2fd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:42.902062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993addd-9cff-45c0-9ea0-c028d5b7021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:42.903232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993addd-9cff-45c0-9ea0-c028d5b7021d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:42.903748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993addd-9cff-45c0-9ea0-c028d5b7021d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:42.904265 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993addd-9cff-45c0-9ea0-c028d5b7021d None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:42.905781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993addd-9cff-45c0-9ea0-c028d5b7021d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:42.906330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993addd-9cff-45c0-9ea0-c028d5b7021d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:42.909166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993addd-9cff-45c0-9ea0-c028d5b7021d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:42.928549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 150/300] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:42.931110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530a651-d260-4070-bb16-167c24e3dcf5 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:42.931623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530a651-d260-4070-bb16-167c24e3dcf5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:42.932116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530a651-d260-4070-bb16-167c24e3dcf5 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:42.935473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 151/301] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:42.942831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530a651-d260-4070-bb16-167c24e3dcf5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:42.947208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fc443-aa7e-4701-97f0-309efece70e4 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:43.040448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d502569-1154-4622-b994-29f002a84638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:43.041955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502569-1154-4622-b994-29f002a84638 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.042555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502569-1154-4622-b994-29f002a84638 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.042950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502569-1154-4622-b994-29f002a84638 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.045433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d502569-1154-4622-b994-29f002a84638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:43.048253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d502569-1154-4622-b994-29f002a84638 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:43.056989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 151/302] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:43.056989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675835ab-05be-4db5-aef3-618699b43a4e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.056989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675835ab-05be-4db5-aef3-618699b43a4e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.056989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675835ab-05be-4db5-aef3-618699b43a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.096884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-675835ab-05be-4db5-aef3-618699b43a4e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:43.099273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-675835ab-05be-4db5-aef3-618699b43a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:43.108551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:43.180519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530a651-d260-4070-bb16-167c24e3dcf5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:43.248031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaacd8b-3ed9-4231-b8bd-c4a0b04c8819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:43.249707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaacd8b-3ed9-4231-b8bd-c4a0b04c8819 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.250142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaacd8b-3ed9-4231-b8bd-c4a0b04c8819 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.250506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaacd8b-3ed9-4231-b8bd-c4a0b04c8819 None admin] PATH_INFO: `/v3/users/87e216781c094f19ae835e3a313712d0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.257179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:43.274281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 152/303] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:43.275602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675835ab-05be-4db5-aef3-618699b43a4e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:43.294474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaacd8b-3ed9-4231-b8bd-c4a0b04c8819 None admin] RBAC: Authorizing `identity:get_user(user_id=87e216781c094f19ae835e3a313712d0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:43.297231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaacd8b-3ed9-4231-b8bd-c4a0b04c8819 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:43.313897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 152/304] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:42 2022] GET /identity/v3/users/87e216781c094f19ae835e3a313712d0 => generated 293 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:43.313897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c100de-cd69-48a0-9fe6-90931dbc8cbf None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.313897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c100de-cd69-48a0-9fe6-90931dbc8cbf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.313897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c100de-cd69-48a0-9fe6-90931dbc8cbf None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.313897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 153/305] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:43.314472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1efa98-d652-46c7-bc0f-bc0d21e64e9e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.314675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1efa98-d652-46c7-bc0f-bc0d21e64e9e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.314999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1efa98-d652-46c7-bc0f-bc0d21e64e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.342802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9f1efa98-d652-46c7-bc0f-bc0d21e64e9e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:43.344948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9f1efa98-d652-46c7-bc0f-bc0d21e64e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:43.447487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675835ab-05be-4db5-aef3-618699b43a4e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:43.458882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 154/306] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:43.459837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1efa98-d652-46c7-bc0f-bc0d21e64e9e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:43.516454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:43.517935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.518708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.519785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] PATH_INFO: `/v3/groups` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.520769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:43.522740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:43.547807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 153/307] 10.222.0.72 () {60 vars in 1134 bytes} [Thu Jun 16 14:40:43 2022] POST /identity/v3/groups => generated 229 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:40:43.549156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:43.613481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:43.702703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:43.703184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1efa98-d652-46c7-bc0f-bc0d21e64e9e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:43.798510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:43.798510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.800505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.800950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.802293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9f4dd-03c1-4303-a8f8-29d25fbefe41 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:43.827721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:43.847890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:43.850405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:43.851848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 155/308] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:43.855281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:43.894125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:43.894663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:43.894949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:43.895267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:43.897323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] RBAC: Adding query filter params (role.id=ad5c06a55fab4b96836a70829887c664, user.id=87e216781c094f19ae835e3a313712d0, scope.system=all) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:43.898089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:43.899806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:43.907527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 154/309] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Jun 16 14:40:43 2022] GET /identity/v3/role_assignments?user.id=87e216781c094f19ae835e3a313712d0&scope.system=all&role.id=ad5c06a55fab4b96836a70829887c664 => generated 226 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:43.908405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:44.092948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e63b9-95ab-4b1b-b0dd-7ab25ed9400d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:44.102800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8f32-b3ec-4c85-bc72-0a17e8bb2910 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:44.142996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:44.156988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:44.174232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:44.176257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:44.177307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:44.178573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:44.186067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:44.187482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:44.188082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:44.188672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:44.190305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:44.190958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:44.195059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:44.203462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:44.218527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:44.219976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:44.221377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:44.222522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 155/310] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:44.223831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:44.226829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 156/311] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:44.237236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:44.431964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea667-f788-4247-ac96-96bd8f0bd5b4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:44.449394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204267d0-f647-42d9-971c-a35844030bcd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:44.522217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:44.523279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:44.523846 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:44.524475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:44.525531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:44.525719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:44.526666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:44.527520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:44.528074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:44.528819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] PATH_INFO: `/v3/users/db44c711e4814b1c9c62eb7c2cb6b812` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:44.529669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:44.531466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] RBAC: Authorizing `identity:get_user(user_id=db44c711e4814b1c9c62eb7c2cb6b812)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:44.535317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:44.545162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 156/312] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:44 2022] GET /identity/v3/users/db44c711e4814b1c9c62eb7c2cb6b812 => generated 293 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:44.548551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:44.555683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 157/313] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:44.566529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:44.669992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:44.677751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:44.795418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc9dce-810a-4a1a-a08e-7de6de77f64c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:44.813341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f112f-5bab-4f02-ac92-d37e2f4ce087 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:44.892204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:44.894334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:44.895298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:44.896691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] PATH_INFO: `/v3/system/users/db44c711e4814b1c9c62eb7c2cb6b812/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:44.900826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=db44c711e4814b1c9c62eb7c2cb6b812, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:44.903006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:44.904508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:44.904806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:44.905530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:44.906070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] PATH_INFO: `/v3/users/9a516dc1cad045019d0d57a0efa826ad` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:44.909239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] RBAC: Authorizing `identity:get_user(user_id=9a516dc1cad045019d0d57a0efa826ad)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:44.912920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:44.924626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 158/314] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:44 2022] GET /identity/v3/users/9a516dc1cad045019d0d57a0efa826ad => generated 297 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:44.928795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:44.940357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 157/315] 10.222.0.72 () {58 vars in 1334 bytes} [Thu Jun 16 14:40:44 2022] PUT /identity/v3/system/users/db44c711e4814b1c9c62eb7c2cb6b812/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:40:45.150865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea72ba-4a3f-404c-a0b7-fa58a52eaab1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:45.182573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:45.228347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:45.229587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:45.230120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:45.230584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:45.388889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:45.392085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:45.603261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 159/316] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:45.613021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:45.806982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad83fd3-b8a9-4079-9881-f7acebe59c73 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:45.835092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:45.878696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9ec29-d5a8-4f56-a9b8-debd59b98a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:45.881432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ec29-d5a8-4f56-a9b8-debd59b98a6e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:45.882369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ec29-d5a8-4f56-a9b8-debd59b98a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:45.883175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ec29-d5a8-4f56-a9b8-debd59b98a6e None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:45.886179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9ec29-d5a8-4f56-a9b8-debd59b98a6e None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:45.892883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9ec29-d5a8-4f56-a9b8-debd59b98a6e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:45.893875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 158/317] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:45 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:45.904358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:45.932482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:46.115804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddf347-7eb0-4b48-81f2-c49cda3b8451 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:46.188049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170178d-24b9-477e-8ae0-80df9d8dfd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:46.189234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170178d-24b9-477e-8ae0-80df9d8dfd32 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:46.189690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170178d-24b9-477e-8ae0-80df9d8dfd32 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:46.190196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170178d-24b9-477e-8ae0-80df9d8dfd32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:46.191909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170178d-24b9-477e-8ae0-80df9d8dfd32 None admin] RBAC: Adding query filter params (role.id=ad5c06a55fab4b96836a70829887c664, scope.project.id=e9cdaf5401b6417a87a138d155210867, user.id=9a516dc1cad045019d0d57a0efa826ad) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:46.192488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170178d-24b9-477e-8ae0-80df9d8dfd32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:46.196068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170178d-24b9-477e-8ae0-80df9d8dfd32 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:46.209341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 160/318] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:40:45 2022] GET /identity/v3/role_assignments?user.id=9a516dc1cad045019d0d57a0efa826ad&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=ad5c06a55fab4b96836a70829887c664 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:47.093397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd31f98-a849-4a08-830e-c9cf9f0a598c None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:47.093948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd31f98-a849-4a08-830e-c9cf9f0a598c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:47.093948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd31f98-a849-4a08-830e-c9cf9f0a598c None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:47.096297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 159/319] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:47.105532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ca6d-2039-4751-9fd3-3ecaceda2cb8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:47.106158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ca6d-2039-4751-9fd3-3ecaceda2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:47.106500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ca6d-2039-4751-9fd3-3ecaceda2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:47.136054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4842ca6d-2039-4751-9fd3-3ecaceda2cb8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:47.138116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4842ca6d-2039-4751-9fd3-3ecaceda2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:47.337515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 161/320] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:47.379630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd31f98-a849-4a08-830e-c9cf9f0a598c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:47.624971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd31f98-a849-4a08-830e-c9cf9f0a598c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:47.678407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d60da-17a4-453c-899b-a854c4883676 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:47.678968 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d60da-17a4-453c-899b-a854c4883676 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:47.679484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d60da-17a4-453c-899b-a854c4883676 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:47.682125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 162/321] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:47.691159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179c755-ccbb-45cc-95d1-dd44c5e30841 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:47.691944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179c755-ccbb-45cc-95d1-dd44c5e30841 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:47.692402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179c755-ccbb-45cc-95d1-dd44c5e30841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:47.702920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:47.705187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:47.705530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:47.705776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:47.707797 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:47.710082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:47.712357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 160/322] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:40:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:47.713610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:47.719219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:47.732681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e179c755-ccbb-45cc-95d1-dd44c5e30841 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:47.736406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e179c755-ccbb-45cc-95d1-dd44c5e30841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:47.851240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:47.907288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 163/323] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:47.942567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179c755-ccbb-45cc-95d1-dd44c5e30841 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:47.974341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efb744-fd64-4923-b34c-fb6271735c75 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:48.040325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:48.041266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:48.041692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:48.042316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] PATH_INFO: `/v3/groups` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:48.043395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:48.045591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:48.077123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 161/324] 10.222.0.72 () {60 vars in 1134 bytes} [Thu Jun 16 14:40:47 2022] POST /identity/v3/groups => generated 226 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:40:48.154422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179c755-ccbb-45cc-95d1-dd44c5e30841 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:48.215039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e229b7-f3f9-4d9e-9178-d88ef3eafd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:48.216392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e229b7-f3f9-4d9e-9178-d88ef3eafd09 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:48.216894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e229b7-f3f9-4d9e-9178-d88ef3eafd09 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:48.217382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e229b7-f3f9-4d9e-9178-d88ef3eafd09 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:48.222447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:48.245745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e229b7-f3f9-4d9e-9178-d88ef3eafd09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-54e229b7-f3f9-4d9e-9178-d88ef3eafd09 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:48.261523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:48.262664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:48.263529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:48.265040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 164/325] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:40:48.274114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:48.507216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eee44a-3451-4915-b2a0-400e68c67c6b None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:48.543093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:48.566610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede01a0-ac94-49fe-b19a-115ff12112d1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:48.567182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede01a0-ac94-49fe-b19a-115ff12112d1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:48.567620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede01a0-ac94-49fe-b19a-115ff12112d1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:48.571324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 165/326] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:48.581991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d18d-199f-4ddc-902e-e620ddcd11b2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:48.582467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d18d-199f-4ddc-902e-e620ddcd11b2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:48.582951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d18d-199f-4ddc-902e-e620ddcd11b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:48.594839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:48.596331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:48.597089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:48.597783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:48.599406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:48.600170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:48.602928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:48.616099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-aa57d18d-199f-4ddc-902e-e620ddcd11b2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:48.618226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-aa57d18d-199f-4ddc-902e-e620ddcd11b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:48.619306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 162/327] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:48.628985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:48.768392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 166/328] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:48.814735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57d18d-199f-4ddc-902e-e620ddcd11b2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:48.907723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdf873-596f-48f8-8d9b-718aa58e911d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:49.004378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:49.006671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:49.007163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:49.010888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] PATH_INFO: `/v3/users/87e216781c094f19ae835e3a313712d0` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:49.011241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] RBAC: Authorizing `identity:get_user(user_id=87e216781c094f19ae835e3a313712d0)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:49.015456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:49.015821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 163/329] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:48 2022] GET /identity/v3/users/87e216781c094f19ae835e3a313712d0 => generated 293 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:49.027693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:49.062772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57d18d-199f-4ddc-902e-e620ddcd11b2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:49.123309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:49.148045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:49.148568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:49.149146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:49.149655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:49.154110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:49.180217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:49.197769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:49.199559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:49.201219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 167/330] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:49.211707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:49.372404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18875-edb4-4e00-8224-8a8367ea02ee None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:49.476093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:49.478840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:49.479307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:49.479906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] PATH_INFO: `/v3/system/users/87e216781c094f19ae835e3a313712d0/roles/ad5c06a55fab4b96836a70829887c664` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:49.482962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=87e216781c094f19ae835e3a313712d0, role_id=ad5c06a55fab4b96836a70829887c664)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:49.485668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:49.501481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 164/331] 10.222.0.72 () {58 vars in 1334 bytes} [Thu Jun 16 14:40:49 2022] PUT /identity/v3/system/users/87e216781c094f19ae835e3a313712d0/roles/ad5c06a55fab4b96836a70829887c664 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:40:49.520714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a972c-3020-4b27-88d0-0a7fb840bb63 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:49.594624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:49.595856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:49.596226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:49.596674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:49.598097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:49.598597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:49.601200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:49.622337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 168/332] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:49.632956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:49.876881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85635d1d-003f-4643-9dbd-3355600cfd1e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:49.898072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:49.979632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bff98-0dd8-4a3c-b697-c48500cc6a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:49.981285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bff98-0dd8-4a3c-b697-c48500cc6a7e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:49.981798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bff98-0dd8-4a3c-b697-c48500cc6a7e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:49.982300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bff98-0dd8-4a3c-b697-c48500cc6a7e None admin] PATH_INFO: `/v3/users/db44c711e4814b1c9c62eb7c2cb6b812` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:49.984824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bff98-0dd8-4a3c-b697-c48500cc6a7e None admin] RBAC: Authorizing `identity:get_user(user_id=db44c711e4814b1c9c62eb7c2cb6b812)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:49.987670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bff98-0dd8-4a3c-b697-c48500cc6a7e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:49.991161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 165/333] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:49 2022] GET /identity/v3/users/db44c711e4814b1c9c62eb7c2cb6b812 => generated 293 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:50.000659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:50.113542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:50.226233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2e573-e6cb-44a3-8ef6-ce29c03a548e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:50.306090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b9439-d0ec-482b-a195-271ecd840017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:50.308077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9439-d0ec-482b-a195-271ecd840017 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:50.308919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9439-d0ec-482b-a195-271ecd840017 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:50.309682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b9439-d0ec-482b-a195-271ecd840017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:50.311651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9439-d0ec-482b-a195-271ecd840017 None admin] RBAC: Adding query filter params (role.id=b724d12e8d154fc4b71baaf2741624d8, user.id=db44c711e4814b1c9c62eb7c2cb6b812, scope.system=all) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:50.312536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9439-d0ec-482b-a195-271ecd840017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:50.315525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b9439-d0ec-482b-a195-271ecd840017 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:50.348518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 169/334] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Jun 16 14:40:49 2022] GET /identity/v3/role_assignments?user.id=db44c711e4814b1c9c62eb7c2cb6b812&scope.system=all&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 510 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:52.806818 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330cbc-fb7b-497c-ad43-f8c28b5064f1 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:52.808889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330cbc-fb7b-497c-ad43-f8c28b5064f1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:52.809423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330cbc-fb7b-497c-ad43-f8c28b5064f1 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:52.811899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 166/335] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:52.819236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084d341-ba3f-4110-8279-980ce4ca88e2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:52.820454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084d341-ba3f-4110-8279-980ce4ca88e2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:52.821041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084d341-ba3f-4110-8279-980ce4ca88e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:52.860171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5084d341-ba3f-4110-8279-980ce4ca88e2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:52.862131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5084d341-ba3f-4110-8279-980ce4ca88e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:53.013392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 170/336] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:53.049068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330cbc-fb7b-497c-ad43-f8c28b5064f1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:53.201383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330cbc-fb7b-497c-ad43-f8c28b5064f1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:53.245268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:53.246121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:53.246412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:53.246698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:53.252161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791697fa-672d-4e89-9bb9-8a819afa558b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:53.252636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791697fa-672d-4e89-9bb9-8a819afa558b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:53.253150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791697fa-672d-4e89-9bb9-8a819afa558b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:53.260048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:53.261472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 171/337] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:53.269899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:53.273463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:53.277099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:53.285669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff08cc-4c63-43ad-8456-5db9d2c951a1 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:53.285669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff08cc-4c63-43ad-8456-5db9d2c951a1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:53.286145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 167/338] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:53.286429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff08cc-4c63-43ad-8456-5db9d2c951a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:53.286660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:53.314217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-acff08cc-4c63-43ad-8456-5db9d2c951a1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:53.315587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-acff08cc-4c63-43ad-8456-5db9d2c951a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:53.378803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:53.419789 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:53.508249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 172/339] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:40:53.559094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff08cc-4c63-43ad-8456-5db9d2c951a1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:53.564548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939169e-0a27-42f8-9791-4592285debfd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:53.680989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:53.682182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:53.682727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:53.683137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:53.685389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:53.685389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:53.687926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:53.716078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 168/340] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:40:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:53.725738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:53.851755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff08cc-4c63-43ad-8456-5db9d2c951a1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:53.970805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:53.972094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:53.972640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:53.973148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:54.006301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:54.016280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:54.022408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:54.023932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:54.028687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 173/341] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:54.038415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:54.069467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:54.097684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638692f1-e586-4611-ab80-7323a597fb69 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:54.116491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:54.251860 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55314d-15f5-42ce-8961-cbd772d194cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:54.253067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55314d-15f5-42ce-8961-cbd772d194cd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:54.253580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55314d-15f5-42ce-8961-cbd772d194cd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:54.254084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55314d-15f5-42ce-8961-cbd772d194cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:54.325454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f8a95-c5bd-4394-95f9-f2a8fa144322 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:54.421082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977470f-aadf-45fb-bd86-72163f9b2417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:54.422163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977470f-aadf-45fb-bd86-72163f9b2417 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:54.422739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977470f-aadf-45fb-bd86-72163f9b2417 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:54.423229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977470f-aadf-45fb-bd86-72163f9b2417 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:54.424814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977470f-aadf-45fb-bd86-72163f9b2417 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:54.425387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977470f-aadf-45fb-bd86-72163f9b2417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:54.428140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977470f-aadf-45fb-bd86-72163f9b2417 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:54.448657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 174/342] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:54.452650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5ea99-1a9d-4416-a99c-81b8328a2fd1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:54.453181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5ea99-1a9d-4416-a99c-81b8328a2fd1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:54.453668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5ea99-1a9d-4416-a99c-81b8328a2fd1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:54.456278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 175/343] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:54.457972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5ea99-1a9d-4416-a99c-81b8328a2fd1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:54.487132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55314d-15f5-42ce-8961-cbd772d194cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:54.489176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55314d-15f5-42ce-8961-cbd772d194cd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:54.605372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:54.623139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5ea99-1a9d-4416-a99c-81b8328a2fd1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:54.665185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a018779-1568-43f7-816d-c478f949b261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:54.666229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a018779-1568-43f7-816d-c478f949b261 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:54.666743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a018779-1568-43f7-816d-c478f949b261 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:54.667256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a018779-1568-43f7-816d-c478f949b261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:54.693758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:54.704527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 169/344] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 980 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:54.709119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdba91-aa28-4093-bcff-99fb259e1939 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:54.709632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdba91-aa28-4093-bcff-99fb259e1939 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:54.710100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdba91-aa28-4093-bcff-99fb259e1939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:54.745577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7ccdba91-aa28-4093-bcff-99fb259e1939 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:54.747962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7ccdba91-aa28-4093-bcff-99fb259e1939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:54.757949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a018779-1568-43f7-816d-c478f949b261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:54.759842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a018779-1568-43f7-816d-c478f949b261 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:54.887366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:54.905658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 176/345] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:54.907303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8cff0-bedf-41bb-88c5-d302edc6691d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:54.907708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8cff0-bedf-41bb-88c5-d302edc6691d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:54.908103 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8cff0-bedf-41bb-88c5-d302edc6691d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:54.910235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 177/346] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:54.912197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8cff0-bedf-41bb-88c5-d302edc6691d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:54.914608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 170/347] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:54.916584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdba91-aa28-4093-bcff-99fb259e1939 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:55.053718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8cff0-bedf-41bb-88c5-d302edc6691d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:55.095883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4065db3-76e7-484c-aa34-5fbffc9401a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:55.096995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4065db3-76e7-484c-aa34-5fbffc9401a8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.097393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4065db3-76e7-484c-aa34-5fbffc9401a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.098144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4065db3-76e7-484c-aa34-5fbffc9401a8 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.100826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4065db3-76e7-484c-aa34-5fbffc9401a8 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:55.103200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4065db3-76e7-484c-aa34-5fbffc9401a8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:55.106039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 178/348] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:54 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:55.107488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4ccc-c0da-4cc0-b805-6f2e7ac8935b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.107796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4ccc-c0da-4cc0-b805-6f2e7ac8935b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.108061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4ccc-c0da-4cc0-b805-6f2e7ac8935b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.131207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdba91-aa28-4093-bcff-99fb259e1939 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:55.133267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-377f4ccc-c0da-4cc0-b805-6f2e7ac8935b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:55.134864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-377f4ccc-c0da-4cc0-b805-6f2e7ac8935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:55.173371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:55.186030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:55.187005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.187311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.187568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.188937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:55.190455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:55.192296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 171/349] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:54 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:55.193235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:55.244202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:55.272289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 179/350] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:55.273859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8010985-5c4d-41ce-a620-7a68407f8afe None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.274215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8010985-5c4d-41ce-a620-7a68407f8afe None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.274470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8010985-5c4d-41ce-a620-7a68407f8afe None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.278042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 180/351] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:55.280078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8010985-5c4d-41ce-a620-7a68407f8afe None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:55.359935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1342bf9-4145-4602-afe2-e8b72176d7e5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:55.412833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:55.413700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.413991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.414273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.433219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:55.445258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:55.447068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 172/352] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:55.449073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:55.468661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8010985-5c4d-41ce-a620-7a68407f8afe None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:55.531959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c6dac-8171-49af-aae3-728d1dff9e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:55.532860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c6dac-8171-49af-aae3-728d1dff9e03 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.533151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c6dac-8171-49af-aae3-728d1dff9e03 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.533425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c6dac-8171-49af-aae3-728d1dff9e03 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.543567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:55.544556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c6dac-8171-49af-aae3-728d1dff9e03 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:55.547008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c6dac-8171-49af-aae3-728d1dff9e03 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:55.551592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 181/353] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:40:55 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:55.551836 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ac5d4-9227-4d04-9694-f75af1c99da3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.552793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ac5d4-9227-4d04-9694-f75af1c99da3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.553329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ac5d4-9227-4d04-9694-f75af1c99da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.579228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f90ac5d4-9227-4d04-9694-f75af1c99da3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:55.580463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f90ac5d4-9227-4d04-9694-f75af1c99da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:55.621696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf18a-e13a-441a-bfb0-16bccf75c1a8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:55.656664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:55.675454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:55.676644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.677025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.677994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.679808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:55.681635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:55.685066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 173/354] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:40:55 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:55.686696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:55.689061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 182/355] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:55.691012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2eefc-c118-4ba9-a8ca-3edb63508ec9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.691369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2eefc-c118-4ba9-a8ca-3edb63508ec9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.692556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2eefc-c118-4ba9-a8ca-3edb63508ec9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.701366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 183/356] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:55.701366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2eefc-c118-4ba9-a8ca-3edb63508ec9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:55.858719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2eefc-c118-4ba9-a8ca-3edb63508ec9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:55.864039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b7aea-ede2-4fbc-9060-ff4798e2b074 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:55.903368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:55.919802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:55.920652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.921027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.925362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.936986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:55.938202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:55.938725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:55.939236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:55.940693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:55.941365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:55.941997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:55.943667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:55.957480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:55.959641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 174/357] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:55.961620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:55.964257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 184/358] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:55.967480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:56.040830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:56.139359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06d43-b932-4366-a888-7292c922f927 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:56.149685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e277145-1d95-46d0-98cf-a792c012dc74 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:56.189219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1fee2-64d5-41bd-9154-b3ecbc5dd465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:56.189908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1fee2-64d5-41bd-9154-b3ecbc5dd465 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:56.190195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1fee2-64d5-41bd-9154-b3ecbc5dd465 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:56.190473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1fee2-64d5-41bd-9154-b3ecbc5dd465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:56.191334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1fee2-64d5-41bd-9154-b3ecbc5dd465 None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=d7f352c1af1440d09ff65044ae139a10) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:56.191653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1fee2-64d5-41bd-9154-b3ecbc5dd465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:56.193790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1fee2-64d5-41bd-9154-b3ecbc5dd465 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:56.201941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 175/359] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:40:55 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:56.203696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9935d24-2b08-49f5-9f14-f3df7fa2efe8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:56.204127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9935d24-2b08-49f5-9f14-f3df7fa2efe8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:56.204537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9935d24-2b08-49f5-9f14-f3df7fa2efe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:56.216302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:56.217445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:56.217914 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:56.218342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:56.219781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=d7f352c1af1440d09ff65044ae139a10) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:56.220274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:56.222823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:56.232753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 185/360] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:40:55 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:56.234707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:56.246845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c9935d24-2b08-49f5-9f14-f3df7fa2efe8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:56.250702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c9935d24-2b08-49f5-9f14-f3df7fa2efe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:56.359874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:56.426529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 176/361] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:56.428858 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9935d24-2b08-49f5-9f14-f3df7fa2efe8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:56.498831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b603aa3-f868-4d81-ac57-323c1e49710f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:56.526360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:56.588669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:56.590031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:56.590474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:56.590906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:56.615827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:56.624238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:56.626072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 186/362] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:56.627653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:56.666789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9935d24-2b08-49f5-9f14-f3df7fa2efe8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:56.735913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:56.737591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:56.738379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:56.739159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:56.741557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:56.742497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:56.745505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:56.763592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 177/363] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:40:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:56.769762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:56.787638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f3c55-276b-46cb-b443-80b82309e3a0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:56.838468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:56.839646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:56.840013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:56.840487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:56.856974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:56.920672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:56.976305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398be7df-6853-41c9-b321-0955e52bac9b None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:56.996276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:56.998801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:57.043236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:57.045015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:57.045784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:57.046509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:57.066276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:40:57.080970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:40:57.082712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:40:57.084387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:40:57.088973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 178/364] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:40:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:40:57.090887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:57.156703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 187/365] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:57.157707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:57.238481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:57.281465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710465-655e-4d14-9d46-3494a23a6a7b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:57.300650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:57.310184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8689634-31a6-4c0f-ba5d-92710d2b0050 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:57.322361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:57.323033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:57.323316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:57.323596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:57.383233 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:57.385158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:57.385989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:57.386778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:57.388577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:57.389495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:57.392726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:57.408636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:57.410342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:57.412722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 179/366] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:40:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:57.414957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:57.494509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:57.547136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 188/367] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:57.547136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:57.645050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc9802-754a-420d-964c-3d18f68bb786 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:57.665422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ac5b-f58b-427b-8106-7b82939a783e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:57.708527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:57.709471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:57.709763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:57.710025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:57.711448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:57.712981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:57.715140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 189/368] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:57 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:57.716096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:57.724872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:57.726748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:57.727614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:57.728401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:57.730498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:57.731527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:57.734892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:57.753616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 180/369] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:57.756288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:57.758546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:57.814045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:57.880499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955578a-6d76-458e-ba2f-6b5a875e24ab None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:57.952590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b870c-f709-4b33-a34d-e406456e8aef None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:57.959223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:57.960433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:57.960996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:57.961484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] PATH_INFO: `/v3/users/87e216781c094f19ae835e3a313712d0` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:57.963842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] RBAC: Authorizing `identity:get_user(user_id=87e216781c094f19ae835e3a313712d0)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:57.966546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:57.970586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 190/370] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:40:57 2022] GET /identity/v3/users/87e216781c094f19ae835e3a313712d0 => generated 293 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:57.972073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:58.024507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:58.026633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:58.027551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:58.028344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:58.030867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:58.034097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:58.038457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 181/371] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:57 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:58.046649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:58.181766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcadcb-8e40-4d2a-89eb-ecf8f7f6a9ed None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:58.184684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:58.239701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:58.253708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:58.255358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:58.255838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:58.256313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:58.258746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:58.261420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:58.263708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658624a-8084-42c4-8353-62683cccdc16 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:58.264516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 191/372] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:40:57 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:58.265971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:58.331982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:58.335009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:58.335904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:58.336757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:58.466553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff8b79-a70f-4421-bacb-d4afc8952331 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:58.474758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:58.477948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:58.534096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:58.535166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:58.535646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:58.536122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:58.537827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] RBAC: Adding query filter params (role.id=ad5c06a55fab4b96836a70829887c664, user.id=87e216781c094f19ae835e3a313712d0, scope.system=all) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:58.538431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:58.541409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:58.564771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 192/373] 10.222.0.72 () {56 vars in 1308 bytes} [Thu Jun 16 14:40:58 2022] GET /identity/v3/role_assignments?user.id=87e216781c094f19ae835e3a313712d0&scope.system=all&role.id=ad5c06a55fab4b96836a70829887c664 => generated 510 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:58.566242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:58.623583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:58.667272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:58.698978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 182/374] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:40:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:40:58.700974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:58.789879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a865bd-191a-4e0d-9ac9-52e3332a76d0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:58.872540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:58.874144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:58.874656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:58.875155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:58.877564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:58.880591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:58.884216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 193/375] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:40:58 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:58.886667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:58.980029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a83a1-0d17-4dea-a18e-d5d4f033c07d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:59.049115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:59.050284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.050778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.051250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.052869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=e9cdaf5401b6417a87a138d155210867) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:59.053467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:59.056424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:59.069592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 183/376] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:40:58 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 260 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:59.071049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:59.082723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:59.095803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be455d42-0ce8-4987-beca-9b790ce624f0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:59.172767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17809cab-9cee-40d0-9d5a-be76fad86a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:59.174726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17809cab-9cee-40d0-9d5a-be76fad86a84 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.175505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17809cab-9cee-40d0-9d5a-be76fad86a84 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.176223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17809cab-9cee-40d0-9d5a-be76fad86a84 None admin] PATH_INFO: `/v3/projects/1304e58d14b64959b1e4ffc146293183` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.178990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17809cab-9cee-40d0-9d5a-be76fad86a84 None admin] RBAC: Authorizing `identity:get_project(project_id=1304e58d14b64959b1e4ffc146293183)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:59.182800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17809cab-9cee-40d0-9d5a-be76fad86a84 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:59.187118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 194/377] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:40:58 2022] GET /identity/v3/projects/1304e58d14b64959b1e4ffc146293183 => generated 344 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:59.195651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067db84e-80e7-47d8-b45e-9db8f101579a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.196489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067db84e-80e7-47d8-b45e-9db8f101579a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.197221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067db84e-80e7-47d8-b45e-9db8f101579a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.200010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 195/378] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:59.204439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067db84e-80e7-47d8-b45e-9db8f101579a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:59.210476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:59.320366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97b5f8-3df5-484d-803e-8d2fff228d18 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:59.393057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885591d-549c-4b06-9bdf-1b65fe650b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:59.396595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885591d-549c-4b06-9bdf-1b65fe650b65 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.397435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885591d-549c-4b06-9bdf-1b65fe650b65 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.398095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885591d-549c-4b06-9bdf-1b65fe650b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.400101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885591d-549c-4b06-9bdf-1b65fe650b65 None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=e9cdaf5401b6417a87a138d155210867) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:59.400914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885591d-549c-4b06-9bdf-1b65fe650b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:59.403812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885591d-549c-4b06-9bdf-1b65fe650b65 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:59.415719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 184/379] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:40:59 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 260 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:59.418587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1796b-0098-44e2-8738-49a060842cc5 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.419433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1796b-0098-44e2-8738-49a060842cc5 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.420159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1796b-0098-44e2-8738-49a060842cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.446526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067db84e-80e7-47d8-b45e-9db8f101579a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:59.450435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-38c1796b-0098-44e2-8738-49a060842cc5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:59.454503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38c1796b-0098-44e2-8738-49a060842cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:40:59.521666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:59.522923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.523417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.523981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] PATH_INFO: `/v3/groups/3ce91b02bd1e42f182abbde950546f3b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.530672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] RBAC: Authorizing `identity:get_group(group_id=3ce91b02bd1e42f182abbde950546f3b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:59.532569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:59.540151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 196/380] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:40:59 2022] GET /identity/v3/groups/3ce91b02bd1e42f182abbde950546f3b => generated 229 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:40:59.542939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:59.658938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:59.707180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 185/381] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:40:59.761402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b377259-6b80-41d1-8077-b1f708ae67df None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.761934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b377259-6b80-41d1-8077-b1f708ae67df None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.762440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b377259-6b80-41d1-8077-b1f708ae67df None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.767147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 186/382] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:40:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:40:59.770937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b377259-6b80-41d1-8077-b1f708ae67df None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:40:59.828887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf642987-8d8d-4a19-8de4-f33f204b841f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:40:59.873538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d799418-a13f-4844-b689-f812bc41467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:40:59.874242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d799418-a13f-4844-b689-f812bc41467c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.874622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d799418-a13f-4844-b689-f812bc41467c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.874996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d799418-a13f-4844-b689-f812bc41467c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.875965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d799418-a13f-4844-b689-f812bc41467c None admin] RBAC: Adding query filter params (group.id=3ce91b02bd1e42f182abbde950546f3b, role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=1304e58d14b64959b1e4ffc146293183) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:40:59.876296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d799418-a13f-4844-b689-f812bc41467c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:40:59.878042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d799418-a13f-4844-b689-f812bc41467c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:40:59.890207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 197/383] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:40:59 2022] GET /identity/v3/role_assignments?group.id=3ce91b02bd1e42f182abbde950546f3b&scope.project.id=1304e58d14b64959b1e4ffc146293183&role.id=c21668b08994436ca32efab183c5ca03 => generated 260 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:40:59.891465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdef619-102f-405a-b92e-f2958744fb96 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:40:59.892042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdef619-102f-405a-b92e-f2958744fb96 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:40:59.892463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdef619-102f-405a-b92e-f2958744fb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:40:59.896009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:40:59.918114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0cdef619-102f-405a-b92e-f2958744fb96 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:40:59.919249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0cdef619-102f-405a-b92e-f2958744fb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:00.027409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 198/384] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:00.038262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b377259-6b80-41d1-8077-b1f708ae67df None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:00.075788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdef619-102f-405a-b92e-f2958744fb96 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:00.121426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:00.138031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:00.139566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:00.140055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:00.140580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:00.187136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:00.204998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:00.206617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:00.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:00.212034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 187/385] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:40:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:00.223098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:00.400492 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdef619-102f-405a-b92e-f2958744fb96 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:00.537907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:00.539262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:00.539759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:00.540249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:00.550876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cec47-021c-49ff-9953-611b8becf05c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:00.587974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:00.588395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:00.612286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:00.614144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:00.615758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:00.630827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 199/386] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 559 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:00.632448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:00.690007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:00.691151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:00.691644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:00.692141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:00.694211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:00.694751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:00.697471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:00.732731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 188/387] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:00.742480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:00.869636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:00.947134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a97f7-f94d-4738-a4e1-04468cd94ba1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:01.081469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:01.086596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47383826-2217-4f81-991c-f757f6e7b285 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:01.090242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:01.090734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:01.091225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:01.092683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:01.093274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:01.095934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:01.123693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 200/388] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:41:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 492 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:01.134104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:01.200963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:01.203059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:01.204228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:01.207277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:01.210355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:01.217898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:01.222932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 189/389] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:00 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:01.233661 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:01.337995 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:01.463934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3626-454a-4b20-86fc-31ea79e702d9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:01.535989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:01.537835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:01.538480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:01.539089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:01.539501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23346998-71c1-4fc2-a0bd-03ec14ba5c2c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:01.541710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:01.548875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:01.548875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 201/390] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:01 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:01.558938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:01.606388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:01.676194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:01.680904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:01.684067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:01.684632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:01.689114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:01.691915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:01.696157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 190/391] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:01 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:01.722308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:01.859541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757313-e48e-4245-b599-45ef4bf10ee8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:01.954510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:01.958653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:01.960086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:01.963318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:01.969107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:01.972643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:01.977586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 202/392] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:01 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:02.014936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:02.049523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a2f69-5673-4d37-b41d-e8b67de48a69 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:02.065414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:02.168290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c2716-5bcc-4a2b-b117-aac4e380f04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:02.169966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2716-5bcc-4a2b-b117-aac4e380f04b None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:02.170472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2716-5bcc-4a2b-b117-aac4e380f04b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:02.170962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c2716-5bcc-4a2b-b117-aac4e380f04b None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10/groups/5a2014b5533d46ce9e503c0047e35c78/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:02.179559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c2716-5bcc-4a2b-b117-aac4e380f04b None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f352c1af1440d09ff65044ae139a10, group_id=5a2014b5533d46ce9e503c0047e35c78, role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:02.182375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c2716-5bcc-4a2b-b117-aac4e380f04b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:02.222142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 191/393] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Jun 16 14:41:01 2022] PUT /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10/groups/5a2014b5533d46ce9e503c0047e35c78/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:41:02.282950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:02.342717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972dcb0-c83a-4edf-8e7d-6c49fb26d874 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:02.434259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6002e31-5974-4369-89b5-96fb52b9c2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:02.435870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6002e31-5974-4369-89b5-96fb52b9c2f0 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:02.436344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6002e31-5974-4369-89b5-96fb52b9c2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:02.436830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6002e31-5974-4369-89b5-96fb52b9c2f0 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10/groups/5a2014b5533d46ce9e503c0047e35c78/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:02.440764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6002e31-5974-4369-89b5-96fb52b9c2f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f352c1af1440d09ff65044ae139a10, group_id=5a2014b5533d46ce9e503c0047e35c78, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:02.444271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6002e31-5974-4369-89b5-96fb52b9c2f0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:02.473075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 203/394] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Jun 16 14:41:02 2022] PUT /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10/groups/5a2014b5533d46ce9e503c0047e35c78/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:41:03.153957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfccc8-1f8d-4eff-ab44-1605a5fa96c2 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:03.154955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfccc8-1f8d-4eff-ab44-1605a5fa96c2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:03.154955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfccc8-1f8d-4eff-ab44-1605a5fa96c2 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:03.156418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 192/395] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:03.167280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf5fa-6b63-4f45-862c-17e0041d45b4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:03.167727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf5fa-6b63-4f45-862c-17e0041d45b4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:03.168204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf5fa-6b63-4f45-862c-17e0041d45b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:03.209993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9efaf5fa-6b63-4f45-862c-17e0041d45b4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:03.212205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9efaf5fa-6b63-4f45-862c-17e0041d45b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:03.354714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7a6a5-ccbe-4bd4-b844-7090a82c2953 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:03.355013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7a6a5-ccbe-4bd4-b844-7090a82c2953 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:03.355437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7a6a5-ccbe-4bd4-b844-7090a82c2953 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:03.359050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 193/396] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:03.365663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f6e5-efcf-4d5e-8a8e-97577afced95 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:03.366587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f6e5-efcf-4d5e-8a8e-97577afced95 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:03.367362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f6e5-efcf-4d5e-8a8e-97577afced95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:03.399294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 204/397] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:03.410886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ffb0f6e5-efcf-4d5e-8a8e-97577afced95 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:03.413521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ffb0f6e5-efcf-4d5e-8a8e-97577afced95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:03.473221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaf5fa-6b63-4f45-862c-17e0041d45b4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:03.596575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 194/398] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:03.640036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f6e5-efcf-4d5e-8a8e-97577afced95 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:03.646016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:03.706220 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaf5fa-6b63-4f45-862c-17e0041d45b4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:03.751279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:03.752127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:03.752408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:03.752685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:03.774974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:03.805224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:03.807000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:03.808517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:03.816730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 205/399] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:03.832740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:03.889749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:03.905913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f6e5-efcf-4d5e-8a8e-97577afced95 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:03.972565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c9cc1-f5b5-4be6-8569-87911d2df800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:03.973903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9cc1-f5b5-4be6-8569-87911d2df800 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:03.974394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9cc1-f5b5-4be6-8569-87911d2df800 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:03.974878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9cc1-f5b5-4be6-8569-87911d2df800 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:03.996918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c9cc1-f5b5-4be6-8569-87911d2df800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-bc5c9cc1-f5b5-4be6-8569-87911d2df800 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:04.015348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:04.017408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:04.018677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 195/400] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:41:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:04.039943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744aa48-848d-4b68-8cb5-a47f8da409c9 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.040946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744aa48-848d-4b68-8cb5-a47f8da409c9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.041749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744aa48-848d-4b68-8cb5-a47f8da409c9 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.048905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 196/401] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:04.050958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744aa48-848d-4b68-8cb5-a47f8da409c9 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:04.091108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b845e-4495-425c-9a98-5ef71fa9532e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:04.126544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:04.135580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abd2ed-3b45-4cc7-9a09-c27320d44e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:04.136628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd2ed-3b45-4cc7-9a09-c27320d44e43 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.136952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd2ed-3b45-4cc7-9a09-c27320d44e43 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.137238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abd2ed-3b45-4cc7-9a09-c27320d44e43 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.138348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abd2ed-3b45-4cc7-9a09-c27320d44e43 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:04.138666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abd2ed-3b45-4cc7-9a09-c27320d44e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:04.140159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abd2ed-3b45-4cc7-9a09-c27320d44e43 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:04.159224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 206/402] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:41:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:04.159224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80079ac-8d49-4dcf-b1b4-a5f32b55c1bb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.159224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80079ac-8d49-4dcf-b1b4-a5f32b55c1bb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.159224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80079ac-8d49-4dcf-b1b4-a5f32b55c1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.178188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d80079ac-8d49-4dcf-b1b4-a5f32b55c1bb None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:04.179470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d80079ac-8d49-4dcf-b1b4-a5f32b55c1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:04.283514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744aa48-848d-4b68-8cb5-a47f8da409c9 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:04.307636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 207/403] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:04.309280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80079ac-8d49-4dcf-b1b4-a5f32b55c1bb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:04.357033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:04.358948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.360295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.361446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.364011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:04.365463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:04.370208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:04.394051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 197/404] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:04.395903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:04.447823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:04.480225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80079ac-8d49-4dcf-b1b4-a5f32b55c1bb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:04.528795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:04.552322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:04.554915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.555705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.556406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.559122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:04.562221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:04.571566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 208/405] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:04 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:04.573882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:04.587658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1526233-c144-4ea1-a220-ec4ef3d5c081 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:04.633608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:04.634716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.635193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.635660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.653710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:04.667342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:04.668386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:41:04.669477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:04.672250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 198/406] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:04.673602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:04.782722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47519b0-56e4-43aa-9213-45b06d2b1af8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:04.789393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:04.816464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8142593-13d1-461b-b002-9cd21c5fb543 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:04.855755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:04.857368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.858022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.858531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.861133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:04.864376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:04.867530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:04.868340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 209/407] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:04 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:04.869008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:04.869833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:04.870035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:04.870516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:04.872181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:04.873978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:04.876283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 199/408] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:04 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:04.877467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:05.016564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:05.106173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f4e96-294b-4d88-a176-72fc7bd761d9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:05.122848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489451ac-3aa2-4a6b-a252-ac04a0a08b55 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:05.189930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:05.191901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:05.193256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:05.194059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:05.196406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:05.199448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:05.202815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 200/409] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:04 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:05.204790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:05.206898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:05.208126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:05.208592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:05.209130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:05.211111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:05.212231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:05.216528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:05.238790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 210/410] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:05.240642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:05.243457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:05.425617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47297b7-d0b1-4863-9e76-91f15324714e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:05.463198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e32550-2c1a-430b-8c92-8be18b8c8f74 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:05.496431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:05.522259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:05.523812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:05.524302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:05.524811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867/groups/5a2014b5533d46ce9e503c0047e35c78/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:05.526491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:05.528601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9cdaf5401b6417a87a138d155210867, group_id=5a2014b5533d46ce9e503c0047e35c78, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:05.529061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:05.529829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:05.530580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867/groups/5a2014b5533d46ce9e503c0047e35c78/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:05.531593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:05.535283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] RBAC: Authorizing `identity:create_grant(project_id=e9cdaf5401b6417a87a138d155210867, group_id=5a2014b5533d46ce9e503c0047e35c78, role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:05.538287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:05.557296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 211/411] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Jun 16 14:41:05 2022] PUT /identity/v3/projects/e9cdaf5401b6417a87a138d155210867/groups/5a2014b5533d46ce9e503c0047e35c78/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:05.557504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 201/412] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Jun 16 14:41:05 2022] PUT /identity/v3/projects/e9cdaf5401b6417a87a138d155210867/groups/5a2014b5533d46ce9e503c0047e35c78/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:05.558859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:05.758069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:05.813808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267347aa-1b26-4893-ae9b-715caf2b4529 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:05.898409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b908a-c224-40df-86b7-b8c8dff38b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:05.900061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b908a-c224-40df-86b7-b8c8dff38b2c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:05.900544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b908a-c224-40df-86b7-b8c8dff38b2c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:05.901300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b908a-c224-40df-86b7-b8c8dff38b2c None admin] PATH_INFO: `/v3/groups/3ce91b02bd1e42f182abbde950546f3b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:05.906499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b908a-c224-40df-86b7-b8c8dff38b2c None admin] RBAC: Authorizing `identity:get_group(group_id=3ce91b02bd1e42f182abbde950546f3b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:05.909522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b908a-c224-40df-86b7-b8c8dff38b2c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:05.913951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 202/413] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:05 2022] GET /identity/v3/groups/3ce91b02bd1e42f182abbde950546f3b => generated 229 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:05.924441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:06.004455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee8c64-1dd4-402c-a09c-0377f0a7fd8f None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.005548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee8c64-1dd4-402c-a09c-0377f0a7fd8f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.006080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee8c64-1dd4-402c-a09c-0377f0a7fd8f None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.013511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 203/414] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:06 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:06.019500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f2847-a61a-46db-83ed-a43d5ec9a7a3 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.020001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f2847-a61a-46db-83ed-a43d5ec9a7a3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.021658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f2847-a61a-46db-83ed-a43d5ec9a7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.062976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b29f2847-a61a-46db-83ed-a43d5ec9a7a3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:06.065044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b29f2847-a61a-46db-83ed-a43d5ec9a7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:06.151019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97e855-00ee-4626-b38d-a456dc7f52cd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:06.227602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:06.228540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.228950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.229241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] PATH_INFO: `/v3/projects/1304e58d14b64959b1e4ffc146293183` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.231057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] RBAC: Authorizing `identity:get_project(project_id=1304e58d14b64959b1e4ffc146293183)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:06.232781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:06.235132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 212/415] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:05 2022] GET /identity/v3/projects/1304e58d14b64959b1e4ffc146293183 => generated 344 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:06.237378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 204/416] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:41:06.244019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:06.261777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f774c-6862-4643-b15c-be208956e968 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.262214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f774c-6862-4643-b15c-be208956e968 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.262369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f774c-6862-4643-b15c-be208956e968 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.264944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 205/417] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:41:06.273413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528dbf-cb83-4e8f-8cbe-e65f7b8fd9a5 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.273864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528dbf-cb83-4e8f-8cbe-e65f7b8fd9a5 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.274985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528dbf-cb83-4e8f-8cbe-e65f7b8fd9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.311007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b0528dbf-cb83-4e8f-8cbe-e65f7b8fd9a5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:06.313816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b0528dbf-cb83-4e8f-8cbe-e65f7b8fd9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:06.339695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:06.441572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 206/418] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:06.444447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528dbf-cb83-4e8f-8cbe-e65f7b8fd9a5 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:06.448413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:06.463358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15f054-2b31-472b-882c-98ffdd9704c4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:06.537530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:06.538620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.539008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.539750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] PATH_INFO: `/v3/projects/1304e58d14b64959b1e4ffc146293183/groups/3ce91b02bd1e42f182abbde950546f3b/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.543804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] RBAC: Authorizing `identity:create_grant(project_id=1304e58d14b64959b1e4ffc146293183, group_id=3ce91b02bd1e42f182abbde950546f3b, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:06.545615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:06.576952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 213/419] 10.222.0.72 () {58 vars in 1442 bytes} [Thu Jun 16 14:41:06 2022] PUT /identity/v3/projects/1304e58d14b64959b1e4ffc146293183/groups/3ce91b02bd1e42f182abbde950546f3b/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:06.578090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:06.784897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528dbf-cb83-4e8f-8cbe-e65f7b8fd9a5 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:06.795831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249395a-3d2e-4fab-af5b-9a36ae662817 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:06.816461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:06.899076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:06.902811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.905328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.906189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.933810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034401-b241-4283-b356-49742cee5c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:06.935349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034401-b241-4283-b356-49742cee5c86 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:06.935933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034401-b241-4283-b356-49742cee5c86 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:06.936456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034401-b241-4283-b356-49742cee5c86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:06.940895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:06.959417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:06.961381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:06.962988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:06.964092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db034401-b241-4283-b356-49742cee5c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:06.967338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 214/420] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:06.975635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-db034401-b241-4283-b356-49742cee5c86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:06.984255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:06.986243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:06.987900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 207/421] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:41:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 544 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:06.995731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034401-b241-4283-b356-49742cee5c86 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:07.189269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:07.338954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034401-b241-4283-b356-49742cee5c86 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:07.368529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd8166-b741-43a0-8117-ccb00131ad24 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:07.445706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:07.446804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:07.447306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:07.447770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:07.449195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:07.449754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:07.452763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:07.463061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:07.464230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:07.464733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:07.465241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:07.466706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] RBAC: Adding query filter params (name=member) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:07.467260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:07.470159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:07.494790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 208/422] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:07.502583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:07.509341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 215/423] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:41:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 535 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:07.516170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:07.635771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:07.882141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63782ace-262d-4527-8e35-fcef82ee4782 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:07.901546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bb673-3728-4431-8d5c-9f0958d84959 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:08.001530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:08.010526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:08.011611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:08.012121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:08.012597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:08.043144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:08.044640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:08.045443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:08.046255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:08.272115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:08.280548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:08.283375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:08.287025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:08.373603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:08.465760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 209/424] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 964 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:41:08.478841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:08.528893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 216/425] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1014 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:41:08.535268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:08.562878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:08.707696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775380-6014-4571-a8b9-f7b2655c4c80 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:08.769621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:08.777779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:08.779453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:08.779847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:08.780318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:08.782693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:08.786155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:08.793048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 217/426] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:08 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:08.798391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:08.924385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abbde1-7ce8-46ee-8f1d-b06db64551ce None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:08.961131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca44578-7959-4417-9632-0d6d27561b34 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:09.017930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:09.019514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.020009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.021082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.023451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:09.026353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:09.037155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 218/427] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:08 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:09.043445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:09.079839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:09.081474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.081964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.082456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] PATH_INFO: `/v3/projects/d7f352c1af1440d09ff65044ae139a10` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.087002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] RBAC: Authorizing `identity:get_project(project_id=d7f352c1af1440d09ff65044ae139a10)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:09.089863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:09.107383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 210/428] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:08 2022] GET /identity/v3/projects/d7f352c1af1440d09ff65044ae139a10 => generated 298 bytes in 629 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:09.125389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:09.257261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:09.342084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:09.359948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f129a02-db39-4089-9cbd-ff921487211d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:09.369188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309054be-ad85-4ad4-a2a3-fdab112e63ba None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:09.442295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0ca86-0627-4e78-b00a-692da210f6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:09.443892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ca86-0627-4e78-b00a-692da210f6d5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.444404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ca86-0627-4e78-b00a-692da210f6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.444933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ca86-0627-4e78-b00a-692da210f6d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.446608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0ca86-0627-4e78-b00a-692da210f6d5 None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=d7f352c1af1440d09ff65044ae139a10) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:09.447150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0ca86-0627-4e78-b00a-692da210f6d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:09.453786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0ca86-0627-4e78-b00a-692da210f6d5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:09.471297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 219/429] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:41:09 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 611 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:09.471297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956474d8-0c55-48b0-a9c7-00ba0974ec13 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.471297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956474d8-0c55-48b0-a9c7-00ba0974ec13 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.471297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956474d8-0c55-48b0-a9c7-00ba0974ec13 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.471297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 220/430] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:09.486846 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48737a-daa4-4a09-bbc1-72e752865f6a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.487444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48737a-daa4-4a09-bbc1-72e752865f6a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.488048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48737a-daa4-4a09-bbc1-72e752865f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.501319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:09.503931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.511569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.512692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.518860 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:09.522303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:09.522581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bb48737a-daa4-4a09-bbc1-72e752865f6a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:09.525050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bb48737a-daa4-4a09-bbc1-72e752865f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:09.538669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 211/431] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:09 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:09.551430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:09.645316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 221/432] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:09.646711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e69060-e253-4e2b-8c70-b6118069e0d9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.646996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e69060-e253-4e2b-8c70-b6118069e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.647284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e69060-e253-4e2b-8c70-b6118069e0d9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.653286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 222/433] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:09.654813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ea53-164a-4809-9036-d25f5e0df674 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.655139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ea53-164a-4809-9036-d25f5e0df674 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.655386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ea53-164a-4809-9036-d25f5e0df674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.707084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-cf93ea53-164a-4809-9036-d25f5e0df674 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:09.710140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cf93ea53-164a-4809-9036-d25f5e0df674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:09.807536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43b50-dcf9-4fd4-88de-39e595aea42f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:09.826784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:09.852323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 223/434] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:09.854206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93ea53-164a-4809-9036-d25f5e0df674 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:09.885669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:09.886340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:09.886804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:09.887249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:09.888805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=d7f352c1af1440d09ff65044ae139a10) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:09.889432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:09.892432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:09.906182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 212/435] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:41:09 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=d7f352c1af1440d09ff65044ae139a10&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 611 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:09.907680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:09.988519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:10.060968 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93ea53-164a-4809-9036-d25f5e0df674 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:10.157502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:10.159115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:10.160186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:10.160714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:10.166427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3483b-002e-40ca-9360-ea636fc34be2 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:10.188180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:10.205590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:10.207365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:10.210520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 224/436] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:41:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:10.220897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:10.273284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:10.275530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:10.275985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:10.278614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:10.299759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:10.312650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:10.315822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:10.318129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 213/437] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:10.325449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:10.328458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:10.449386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c2182-8512-4615-b438-eac4539b5aa1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:10.483018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:10.519605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd953b03-cfa8-48ef-8600-0289199184ca None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:10.523880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28d954-c28e-4047-9bf5-6ab20509cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:10.525113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d954-c28e-4047-9bf5-6ab20509cd1e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:10.525616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d954-c28e-4047-9bf5-6ab20509cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:10.526093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d954-c28e-4047-9bf5-6ab20509cd1e None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:10.527835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28d954-c28e-4047-9bf5-6ab20509cd1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:10.528356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28d954-c28e-4047-9bf5-6ab20509cd1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:10.531430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28d954-c28e-4047-9bf5-6ab20509cd1e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:10.547917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 225/438] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:10.550406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677454e-a13d-434c-a607-b6dc1ee7d9ea None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:10.550858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677454e-a13d-434c-a607-b6dc1ee7d9ea None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:10.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677454e-a13d-434c-a607-b6dc1ee7d9ea None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:10.554651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 226/439] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:10.559223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677454e-a13d-434c-a607-b6dc1ee7d9ea None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:10.603331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68b28b-ce34-4681-9806-e5304510fa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:10.605338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68b28b-ce34-4681-9806-e5304510fa54 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:10.605828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68b28b-ce34-4681-9806-e5304510fa54 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:10.606322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68b28b-ce34-4681-9806-e5304510fa54 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:10.607811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68b28b-ce34-4681-9806-e5304510fa54 None admin] RBAC: Adding query filter params (name=member) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:10.608355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68b28b-ce34-4681-9806-e5304510fa54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:10.611121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68b28b-ce34-4681-9806-e5304510fa54 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:10.632164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 214/440] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:41:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:10.634784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d988e4-331b-40f8-aada-5f169978d31e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:10.635633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d988e4-331b-40f8-aada-5f169978d31e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:10.636327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d988e4-331b-40f8-aada-5f169978d31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:10.664925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-52d988e4-331b-40f8-aada-5f169978d31e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:10.667230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-52d988e4-331b-40f8-aada-5f169978d31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:10.761652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677454e-a13d-434c-a607-b6dc1ee7d9ea None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:10.785839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:10.803304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 215/441] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:10.805425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d988e4-331b-40f8-aada-5f169978d31e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:10.838284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:10.839460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:10.839965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:10.840465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:10.960136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:10.963312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:10.966022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:11.006664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d988e4-331b-40f8-aada-5f169978d31e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:11.074705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:11.075839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:11.076334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:11.076833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:11.147840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 227/442] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:11.149034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:11.185565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:11.225969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:11.228912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:11.363425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09d78d-c047-4d36-86d7-9de5ca60f97f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:11.378208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:11.418312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 216/443] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:11.419847 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:11.431221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:11.432206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:11.432585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:11.433002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:11.448006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:11.456994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:11.459577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 228/444] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:41:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:11.461852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:11.571338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:11.615645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dc96e-f26d-415f-b33e-e292afccf09e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:11.658969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a748d77-31b3-46ee-b77c-66e11ef4e9eb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:11.694679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:11.696313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:11.696810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:11.697348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:11.699697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:11.702520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:11.705852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 217/445] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:11 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:11.707278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:11.711090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:11.712496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:11.713394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:11.713555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] PATH_INFO: `/v3/projects/e9cdaf5401b6417a87a138d155210867` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:11.715514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] RBAC: Authorizing `identity:get_project(project_id=e9cdaf5401b6417a87a138d155210867)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:11.717144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:11.719214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 229/446] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:11 2022] GET /identity/v3/projects/e9cdaf5401b6417a87a138d155210867 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:11.729010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:11.808030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:11.921149 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6caaa-53ed-4812-bb06-f3658dbfa850 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:11.924876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bed2f-1879-42f8-b713-c6dbd415287b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:11.984851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:11.986022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:11.986514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:11.987009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:11.988441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:11.989008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:11.990474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:11.991941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:11.992320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:11.992454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:11.992943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:11.995324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:11.998052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:12.001200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 230/447] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:11 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:12.002775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:12.014607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 218/448] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:12.016454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:12.025309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:12.203076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865d5cb-8383-4024-998c-5c8f6ac52fc6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:12.228979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eab770-58e0-48b5-a370-63f932ca01bc None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:12.242701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:12.247504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:12.248386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:12.248666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:12.248974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] PATH_INFO: `/v3/groups/5a2014b5533d46ce9e503c0047e35c78` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:12.250298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] RBAC: Authorizing `identity:get_group(group_id=5a2014b5533d46ce9e503c0047e35c78)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:12.251815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:12.253626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 231/449] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:12 2022] GET /identity/v3/groups/5a2014b5533d46ce9e503c0047e35c78 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:12.254461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:12.283364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:12.284201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:12.284585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:12.284977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:12.286628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=e9cdaf5401b6417a87a138d155210867) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:12.287093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:12.289239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:12.297458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 219/450] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:41:12 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 611 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:12.298926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:12.429512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:12.487946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235c64a-6673-45b3-a467-f96d53831f27 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:12.510540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853f6e-dc13-44d8-887c-123a8cf01df7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:12.567660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:12.568504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:12.568784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:12.569240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:12.572318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:12.573514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:12.574042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:12.574565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:12.576237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] RBAC: Adding query filter params (group.id=5a2014b5533d46ce9e503c0047e35c78, role.id=b724d12e8d154fc4b71baaf2741624d8, scope.project.id=e9cdaf5401b6417a87a138d155210867) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:12.577019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:12.579614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:12.589343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 220/451] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:41:12 2022] GET /identity/v3/role_assignments?group.id=5a2014b5533d46ce9e503c0047e35c78&scope.project.id=e9cdaf5401b6417a87a138d155210867&role.id=b724d12e8d154fc4b71baaf2741624d8 => generated 611 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:12.702514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:12.705467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:12.890521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:12.937221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 232/452] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:12.946298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:12.996953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:13.102348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8084a-3761-4b8d-b5f9-3775f64f47bb None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:13.154330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:13.155358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:13.155827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:13.156260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] PATH_INFO: `/v3/projects/1304e58d14b64959b1e4ffc146293183` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:13.157939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] RBAC: Authorizing `identity:get_project(project_id=1304e58d14b64959b1e4ffc146293183)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:13.159744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:13.165968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 221/453] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:41:12 2022] GET /identity/v3/projects/1304e58d14b64959b1e4ffc146293183 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:13.169925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:13.355433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955e3d8-71b9-4c86-a3e8-8554b8960f9c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:13.412113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b7397-6423-4672-a699-6de56c256f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:13.412963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b7397-6423-4672-a699-6de56c256f1f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:13.413297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b7397-6423-4672-a699-6de56c256f1f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:13.413625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b7397-6423-4672-a699-6de56c256f1f None admin] PATH_INFO: `/v3/groups/3ce91b02bd1e42f182abbde950546f3b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:13.414975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b7397-6423-4672-a699-6de56c256f1f None admin] RBAC: Authorizing `identity:get_group(group_id=3ce91b02bd1e42f182abbde950546f3b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:13.416532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b7397-6423-4672-a699-6de56c256f1f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:13.418314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 233/454] 10.222.0.72 () {56 vars in 1180 bytes} [Thu Jun 16 14:41:13 2022] GET /identity/v3/groups/3ce91b02bd1e42f182abbde950546f3b => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:13.425257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:13.611911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a968f-d061-417a-b92a-28a0e5178b8b None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:13.614379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:13.659656 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20f79b-9a3a-4393-a64b-f4403bb03236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:13.660723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f79b-9a3a-4393-a64b-f4403bb03236 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:13.661138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f79b-9a3a-4393-a64b-f4403bb03236 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:13.661598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20f79b-9a3a-4393-a64b-f4403bb03236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:13.662580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20f79b-9a3a-4393-a64b-f4403bb03236 None admin] RBAC: Adding query filter params (group.id=3ce91b02bd1e42f182abbde950546f3b, role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=1304e58d14b64959b1e4ffc146293183) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:13.662883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20f79b-9a3a-4393-a64b-f4403bb03236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:13.664472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20f79b-9a3a-4393-a64b-f4403bb03236 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:13.672225 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 222/455] 10.222.0.72 () {56 vars in 1376 bytes} [Thu Jun 16 14:41:13 2022] GET /identity/v3/role_assignments?group.id=3ce91b02bd1e42f182abbde950546f3b&scope.project.id=1304e58d14b64959b1e4ffc146293183&role.id=c21668b08994436ca32efab183c5ca03 => generated 611 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:17.402158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baa003-118d-4874-955d-5ea251335e9d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:17.402742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baa003-118d-4874-955d-5ea251335e9d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:17.403096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9baa003-118d-4874-955d-5ea251335e9d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:17.406425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 234/456] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:17.416326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c172-bc5c-40e9-8dd9-9ec7bb4e1aef None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:17.418301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c172-bc5c-40e9-8dd9-9ec7bb4e1aef None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:17.419215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c172-bc5c-40e9-8dd9-9ec7bb4e1aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:17.450842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5c35c172-bc5c-40e9-8dd9-9ec7bb4e1aef None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:17.452074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5c35c172-bc5c-40e9-8dd9-9ec7bb4e1aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:17.478938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076815d4-2aad-464e-91bb-f5c97df84335 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:17.479691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076815d4-2aad-464e-91bb-f5c97df84335 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:17.480219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076815d4-2aad-464e-91bb-f5c97df84335 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:17.482250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 235/457] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:17.488933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670acd1-e331-4888-8c8b-5d981d25a52e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:17.489578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670acd1-e331-4888-8c8b-5d981d25a52e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:17.490064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670acd1-e331-4888-8c8b-5d981d25a52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:17.516356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d670acd1-e331-4888-8c8b-5d981d25a52e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:17.519805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d670acd1-e331-4888-8c8b-5d981d25a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:17.576696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 223/458] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:17.612702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:17.687518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c172-bc5c-40e9-8dd9-9ec7bb4e1aef None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:17.688964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 236/459] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:41:17.749016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670acd1-e331-4888-8c8b-5d981d25a52e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:17.928587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670acd1-e331-4888-8c8b-5d981d25a52e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:17.970961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:17.992146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c172-bc5c-40e9-8dd9-9ec7bb4e1aef None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:17.994884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:17.996266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:17.996749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:17.997088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:18.011973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:18.013829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:18.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:18.040953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:18.042792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:18.042792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:18.042792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:18.042792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:18.042792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 237/460] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:18.047017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:18.094058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:18.154482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:18.155931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:18.156443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:18.157001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:18.190463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:18.193534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:18.245542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:18.247202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:18.249009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:18.249009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:18.249009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:18.249009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:18.263832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 224/461] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 577 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:18.264874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:18.387162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e94bff-2d3c-438f-9713-839ad1ef3405 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:18.496703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5b7ee-853f-4eea-9295-60cfe85726db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:18.500559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5b7ee-853f-4eea-9295-60cfe85726db None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:18.501557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5b7ee-853f-4eea-9295-60cfe85726db None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:18.502375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5b7ee-853f-4eea-9295-60cfe85726db None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:18.504723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5b7ee-853f-4eea-9295-60cfe85726db None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:18.506015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5b7ee-853f-4eea-9295-60cfe85726db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:18.509835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5b7ee-853f-4eea-9295-60cfe85726db None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:18.560878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 238/462] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:18.563515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168e41-dc94-4701-884a-9388e4dbb02c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:18.563905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168e41-dc94-4701-884a-9388e4dbb02c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:18.564497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168e41-dc94-4701-884a-9388e4dbb02c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:18.573507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 239/463] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:18.575186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34168e41-dc94-4701-884a-9388e4dbb02c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:18.584433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f91e8-d7ff-4928-8973-6ccaedcf4947 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:18.640323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ccd3e-e26f-4636-90b6-67be6713c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:18.641393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ccd3e-e26f-4636-90b6-67be6713c4fd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:18.641712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ccd3e-e26f-4636-90b6-67be6713c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:18.642263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ccd3e-e26f-4636-90b6-67be6713c4fd None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:18.643160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ccd3e-e26f-4636-90b6-67be6713c4fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:18.643723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ccd3e-e26f-4636-90b6-67be6713c4fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:18.646952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ccd3e-e26f-4636-90b6-67be6713c4fd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:18.670556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 225/464] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:18.672111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc27e66-5b9b-4e01-98d5-dfd4d81d74ed None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:18.672461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc27e66-5b9b-4e01-98d5-dfd4d81d74ed None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:18.672755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc27e66-5b9b-4e01-98d5-dfd4d81d74ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:18.699784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5fc27e66-5b9b-4e01-98d5-dfd4d81d74ed None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:18.702143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5fc27e66-5b9b-4e01-98d5-dfd4d81d74ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:18.706396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:18.776972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:18.805791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34168e41-dc94-4701-884a-9388e4dbb02c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:18.836729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 226/465] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:18.837811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc27e66-5b9b-4e01-98d5-dfd4d81d74ed None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:18.916667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:18.918328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:18.919074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:18.919611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:18.921513 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:18.925711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:18.986946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 240/466] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:41:18 2022] POST /identity/v3/users => generated 248 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:41:18.989909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:19.052097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc27e66-5b9b-4e01-98d5-dfd4d81d74ed None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:19.140297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a480f6-5850-4143-b4b2-c53375d14b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:19.142292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a480f6-5850-4143-b4b2-c53375d14b0e None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.142521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a480f6-5850-4143-b4b2-c53375d14b0e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.142979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a480f6-5850-4143-b4b2-c53375d14b0e None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.144293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a480f6-5850-4143-b4b2-c53375d14b0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:19.147512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a480f6-5850-4143-b4b2-c53375d14b0e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:19.163861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:19.200923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 227/467] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:41:18 2022] POST /identity/v3/users => generated 246 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:41:19.200923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3d07a-ad0f-4e30-9318-603d2c8f3038 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.200923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3d07a-ad0f-4e30-9318-603d2c8f3038 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.200923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3d07a-ad0f-4e30-9318-603d2c8f3038 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.207429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 228/468] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:19.210092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2118c3-3ff1-4c68-bfe9-e0753196f7cf None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.210572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2118c3-3ff1-4c68-bfe9-e0753196f7cf None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.211100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2118c3-3ff1-4c68-bfe9-e0753196f7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.255688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7d2118c3-3ff1-4c68-bfe9-e0753196f7cf None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:19.258048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7d2118c3-3ff1-4c68-bfe9-e0753196f7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:19.296419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd349f-d93d-4520-8593-f247a74055c4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:19.365316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:19.367528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.368496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.369527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.394059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:19.398195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:19.402644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:19.432976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 229/469] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:19.445581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:19.447322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:19.449205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:19.449205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:19.449205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:19.449205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:19.453455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 241/470] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:19.459939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2118c3-3ff1-4c68-bfe9-e0753196f7cf None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:19.463323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:19.669279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabe0de-0c0f-4367-9ded-23e6f7433b92 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:19.691950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2118c3-3ff1-4c68-bfe9-e0753196f7cf None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:19.740732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee811c5-ce90-410f-beb2-8899b3d7f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:19.741653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee811c5-ce90-410f-beb2-8899b3d7f30a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.742037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee811c5-ce90-410f-beb2-8899b3d7f30a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.742323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee811c5-ce90-410f-beb2-8899b3d7f30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.749482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:19.766156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d800ed6-563b-4551-b1b0-d2b478c72b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:19.767220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d800ed6-563b-4551-b1b0-d2b478c72b86 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.767220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d800ed6-563b-4551-b1b0-d2b478c72b86 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.767546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d800ed6-563b-4551-b1b0-d2b478c72b86 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.768319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d800ed6-563b-4551-b1b0-d2b478c72b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:19.768683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d800ed6-563b-4551-b1b0-d2b478c72b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:19.771785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d800ed6-563b-4551-b1b0-d2b478c72b86 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:19.776572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee811c5-ce90-410f-beb2-8899b3d7f30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:19.779992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee811c5-ce90-410f-beb2-8899b3d7f30a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:19.793740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 230/471] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:19.797376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa95359-9adc-4b22-8ea0-05b79a0601f1 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.797915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa95359-9adc-4b22-8ea0-05b79a0601f1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.798648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa95359-9adc-4b22-8ea0-05b79a0601f1 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.802851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 231/472] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:19.806590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa95359-9adc-4b22-8ea0-05b79a0601f1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3ee811c5-ce90-410f-beb2-8899b3d7f30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:19.825549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:19.827428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:19.829138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:19.829138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:19.829138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:19.829138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:19.836977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 242/473] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:19.839753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7c03-ba0f-432c-a68a-ccfffb42097f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:19.840929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7c03-ba0f-432c-a68a-ccfffb42097f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:19.841774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c7c03-ba0f-432c-a68a-ccfffb42097f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:19.879472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a41c7c03-ba0f-432c-a68a-ccfffb42097f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:19.882268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a41c7c03-ba0f-432c-a68a-ccfffb42097f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:19.970238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:19.998987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 243/474] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:20.002796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c7c03-ba0f-432c-a68a-ccfffb42097f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:20.072083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa95359-9adc-4b22-8ea0-05b79a0601f1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:20.152916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:20.154049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:20.154575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:20.155072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:20.156440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:20.159350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:20.205463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 232/475] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:41:19 2022] POST /identity/v3/users => generated 248 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:41:20.207285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:20.213805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c7c03-ba0f-432c-a68a-ccfffb42097f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:20.309526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:20.354220 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:20.355425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:20.356033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:20.356539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:20.358034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:20.358608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:20.369742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:20.401521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49448922-f248-4697-9afb-2fa39daa4f59 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:20.420180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 244/476] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:20.431186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:20.498311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:20.500194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:20.500609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:20.501063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:20.502303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:20.505015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:20.532506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.sql.core [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 14:41:20.532506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 14:41:20.532506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [parameters: {'id': '2d753ccfd1af46c982a0444a08f5d946', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 14:41:20.532506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86694) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 14:41:20.573645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application result.read() Jun 16 14:41:20.596098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:41:20.597991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application yield resource Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise self.value Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application session.commit() Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 14:41:20.599967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self.session.flush() Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application compat.raise_( Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 14:41:20.603324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:41:20.613477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application result.read() Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:41:20.615999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [parameters: {'id': '2d753ccfd1af46c982a0444a08f5d946', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:20.617985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 14:41:20.619894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:20.622242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 233/477] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Jun 16 14:41:20 2022] POST /identity/v3/roles => generated 144 bytes in 394 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:20.622242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:20.748975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d60ed-3146-4ac7-acee-46aecc79ad89 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:20.796124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e14aa4-4f0b-4def-8f8e-bf26f8c791f8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:20.799292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:20.843888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b21338-9377-4310-883d-cd3bd40523fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:20.845870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b21338-9377-4310-883d-cd3bd40523fa None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:20.846893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b21338-9377-4310-883d-cd3bd40523fa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:20.847745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b21338-9377-4310-883d-cd3bd40523fa None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:20.850574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b21338-9377-4310-883d-cd3bd40523fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:20.850904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:20.851884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:20.852270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:20.852647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:20.854870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b21338-9377-4310-883d-cd3bd40523fa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:20.867178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:20.875901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:20.878141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:20.880047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 234/478] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:41:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:20.892658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:20.906482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 245/479] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:41:20 2022] POST /identity/v3/users => generated 249 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:41:21.091982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073363b-789f-4d3f-b55c-f798eceb96b8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:21.137447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94379349-8641-4f64-962a-004f699d4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:21.138128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94379349-8641-4f64-962a-004f699d4a98 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:21.138645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94379349-8641-4f64-962a-004f699d4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:21.138944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94379349-8641-4f64-962a-004f699d4a98 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:21.139754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94379349-8641-4f64-962a-004f699d4a98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:21.140097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94379349-8641-4f64-962a-004f699d4a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:21.142329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94379349-8641-4f64-962a-004f699d4a98 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:21.158135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 235/480] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:22.949370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a72de8-38b2-4dbb-a27e-799020890a88 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:22.950785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a72de8-38b2-4dbb-a27e-799020890a88 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:22.951502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a72de8-38b2-4dbb-a27e-799020890a88 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:22.955875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 246/481] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:22.961991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94a16-1701-47e3-9f64-667ff19d7a10 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:22.962679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94a16-1701-47e3-9f64-667ff19d7a10 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:22.962972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf94a16-1701-47e3-9f64-667ff19d7a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:22.995256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-edf94a16-1701-47e3-9f64-667ff19d7a10 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:22.996667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-edf94a16-1701-47e3-9f64-667ff19d7a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:23.042011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:23.132098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 236/482] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:23.159089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a72de8-38b2-4dbb-a27e-799020890a88 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:23.325846 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64eb82-7065-45fb-9f1a-698a731fccc0 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:23.326813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64eb82-7065-45fb-9f1a-698a731fccc0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:23.327607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64eb82-7065-45fb-9f1a-698a731fccc0 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:23.336825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 237/483] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:23.340239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48419387-6294-4dfe-a3c2-ae7bac91ce8a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:23.342031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48419387-6294-4dfe-a3c2-ae7bac91ce8a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:23.343047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48419387-6294-4dfe-a3c2-ae7bac91ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:23.352900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:23.375083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a72de8-38b2-4dbb-a27e-799020890a88 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:23.383100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-48419387-6294-4dfe-a3c2-ae7bac91ce8a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:23.385483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-48419387-6294-4dfe-a3c2-ae7bac91ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:23.488159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:23.490261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:23.495660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:23.496501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:23.523720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:23.539654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:23.541260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:23.542600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:23.547011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 247/484] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:23.556309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:23.620875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 238/485] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:41:23.673053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48419387-6294-4dfe-a3c2-ae7bac91ce8a None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:23.811204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db647acd-fe14-4786-87ea-6b3cf44db570 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:23.854694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:23.872273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48419387-6294-4dfe-a3c2-ae7bac91ce8a None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:23.912065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:23.913019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:23.913368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:23.913711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:23.917691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:23.918396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:23.918765 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:23.919052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:23.919863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:23.920190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:23.921676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:23.927649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:23.936679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:23.938208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:23.940163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 239/486] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:23.945733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 248/487] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:23.958579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:23.961937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:24.007639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:24.222393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca630d9-a787-4e9a-a7ef-67026d20c106 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:24.261514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeaa51-54b4-4e6c-8589-62e5b9407163 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:24.361979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:24.363128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:24.363613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:24.364615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:24.366048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:24.366576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:24.369256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:24.373958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:24.375436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:24.375944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:24.376448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:24.398025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:24.400626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:24.403459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 249/488] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:24.412687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:24.436151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:24.468268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:24.469944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:24.471563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:24.471563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:24.471563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:24.471563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:24.485081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 240/489] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 523 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:24.490396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:24.627529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:24.684337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8dc40-c007-4236-a87e-c13b3f3f7d86 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:24.697093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b85b-ec2e-401d-8154-c3925b9ee406 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:24.749071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3a2ac-0ce8-4b0c-b230-adb07206e81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:24.751022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a2ac-0ce8-4b0c-b230-adb07206e81b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:24.751022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a2ac-0ce8-4b0c-b230-adb07206e81b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:24.751022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a2ac-0ce8-4b0c-b230-adb07206e81b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:24.762461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c012a8e-8a76-4b0a-8f04-6428e2af37b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:24.763431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c012a8e-8a76-4b0a-8f04-6428e2af37b9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:24.763851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c012a8e-8a76-4b0a-8f04-6428e2af37b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:24.764317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c012a8e-8a76-4b0a-8f04-6428e2af37b9 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:24.765693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c012a8e-8a76-4b0a-8f04-6428e2af37b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:24.766232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c012a8e-8a76-4b0a-8f04-6428e2af37b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:24.769093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c012a8e-8a76-4b0a-8f04-6428e2af37b9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:24.773122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3a2ac-0ce8-4b0c-b230-adb07206e81b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:24.775198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3a2ac-0ce8-4b0c-b230-adb07206e81b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:24.792815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 241/490] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:24.795539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c590c-a30d-4c9d-a2df-55d9fd4bd037 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:24.796333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c590c-a30d-4c9d-a2df-55d9fd4bd037 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:24.797076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c590c-a30d-4c9d-a2df-55d9fd4bd037 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:24.800314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 242/491] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:24.802885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdc5b1-74bf-45e5-8bdf-95c098585d46 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:24.802885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdc5b1-74bf-45e5-8bdf-95c098585d46 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:24.803349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdc5b1-74bf-45e5-8bdf-95c098585d46 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:24.805352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 243/492] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:24.812139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdc5b1-74bf-45e5-8bdf-95c098585d46 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-73b3a2ac-0ce8-4b0c-b230-adb07206e81b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:24.819434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:24.821189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:24.822686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:24.822686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:24.822686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:24.822686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:24.822686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 250/493] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:24.824218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec784fac-bd29-47d8-9fa5-a1a2fd4295dc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:24.824723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec784fac-bd29-47d8-9fa5-a1a2fd4295dc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:24.825265 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec784fac-bd29-47d8-9fa5-a1a2fd4295dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:24.851679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ec784fac-bd29-47d8-9fa5-a1a2fd4295dc None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:24.853854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ec784fac-bd29-47d8-9fa5-a1a2fd4295dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:24.969105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:25.020992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 251/494] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:25.020992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b12d5-a968-4755-9faf-665d43800e7e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.020992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b12d5-a968-4755-9faf-665d43800e7e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.020992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b12d5-a968-4755-9faf-665d43800e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.048719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fe8b12d5-a968-4755-9faf-665d43800e7e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:25.050857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fe8b12d5-a968-4755-9faf-665d43800e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:25.064202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdc5b1-74bf-45e5-8bdf-95c098585d46 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:25.122899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:25.123855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.124246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.124616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.134562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:25.137275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:25.160468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:25.162856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:41:25.164611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:25.165699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:25.165829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 244/495] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:25.165829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:25.186672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 252/496] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:25.195856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905dd3-d2f9-4711-add7-52ebd214cbb5 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.196314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905dd3-d2f9-4711-add7-52ebd214cbb5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.196818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905dd3-d2f9-4711-add7-52ebd214cbb5 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.199632 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 253/497] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:25.201154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6905dd3-d2f9-4711-add7-52ebd214cbb5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:25.340966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8cca5-aa4a-4a09-ba31-42f4bdf38313 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:25.365609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6905dd3-d2f9-4711-add7-52ebd214cbb5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:25.374939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:25.408618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:25.409544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.410132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.411110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.412007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:25.412470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:25.414070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:25.422725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039b9cf-e429-46aa-b45f-a0d098a7b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:25.424159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039b9cf-e429-46aa-b45f-a0d098a7b534 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.424742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039b9cf-e429-46aa-b45f-a0d098a7b534 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.425274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039b9cf-e429-46aa-b45f-a0d098a7b534 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.430181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 245/498] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:25.431332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:25.443757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039b9cf-e429-46aa-b45f-a0d098a7b534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6039b9cf-e429-46aa-b45f-a0d098a7b534 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:25.457526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:25.458868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:25.459664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:25.460697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 254/499] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:25.462138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705be64e-8dc8-4c04-9503-07116df84778 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.462625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705be64e-8dc8-4c04-9503-07116df84778 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.463083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705be64e-8dc8-4c04-9503-07116df84778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.494004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-705be64e-8dc8-4c04-9503-07116df84778 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:25.495894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-705be64e-8dc8-4c04-9503-07116df84778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:25.571119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86e8fb-1889-4299-8701-1cd5e7a44b21 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:25.580819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:25.619222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 255/500] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:25.620110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705be64e-8dc8-4c04-9503-07116df84778 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:25.630762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:25.631684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.632176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.633943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.634503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:25.635076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:25.637705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:25.704495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 246/501] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Jun 16 14:41:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:25.705654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:25.774964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:25.830358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705be64e-8dc8-4c04-9503-07116df84778 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:25.904509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:25.905822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.906278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.906730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.909562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d63af-9b53-4a1f-ba18-d24c2da639c6 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:25.926459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:25.929124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:25.963959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:25.965664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:25.967057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:25.967057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:25.967057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:25.967057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:25.967057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 256/502] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:25.967775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:25.981592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:25.982887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:25.983347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:25.983980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:25.997197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:25.999695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:26.023638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:26.027306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:26.029288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 247/503] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:41:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:26.031342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:26.144111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527230eb-5e24-4313-b516-10a8e80af055 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:26.186921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:26.187694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:26.187967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:26.188236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:26.189400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:26.189785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:26.191399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:26.203481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 257/504] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:26.204780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:26.213843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:26.224621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fc6f0-b415-4322-b3a4-221a4474e770 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:26.288424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:26.289869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:26.290364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:26.291263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:26.317306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:26.327677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:26.328922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:26.329991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 248/505] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:26.331413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:26.380630 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:26.396794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b1e6f-9a97-4fe1-b1d7-b37fd4288719 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:26.449044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:26.449836 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:26.450173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:26.450559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:26.516557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29ff87-8616-4d2f-9c1e-d0e3fae45aaa None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:26.557113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:26.558805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:26.569460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:26.570150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:26.570440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:26.570825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:26.571477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:26.571798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:26.573291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:26.587710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 249/506] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:26.588585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:26.618256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:26.700095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:26.714132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 258/507] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:26.715803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:26.747697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4457b8e-7e21-466d-b372-a37b1ff4e64f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:26.800063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:26.801673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:26.802134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:26.802586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:26.804041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:26.804548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:26.807186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:26.840889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 250/508] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:41:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:26.842112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:26.874464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08935902-bfda-4260-a452-98d0567ce80f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:26.919380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:26.920293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:26.920669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:26.920975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:26.935819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:26.937394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:26.945690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:26.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:26.959732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:26.961866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:26.961866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:26.961866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:26.961866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:26.961866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 259/509] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:26.961866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:27.000191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b004d8-3309-4e17-a6b3-57728ce7f42a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:27.038507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:27.044819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:27.045548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:27.045845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:27.046125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:27.047000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:27.047328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:27.048760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:27.060228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 251/510] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:27.061890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:27.130710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34916bc8-86d4-420d-93bb-7b5e363f8f9b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:27.183978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:27.185016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:27.185445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:27.185897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:27.187106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:27.189522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:27.219902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e5c34-919e-46a3-8ecd-33a461ac4006 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:27.220465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 260/511] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:41:26 2022] POST /identity/v3/users => generated 247 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:41:27.221955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:27.273186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:27.274609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:27.275053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:27.275470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:27.280286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:27.295810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:27.298299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:27.327579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:27.338084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:27.340028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:27.340028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:27.340028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:27.340028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:27.340028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 252/512] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:27.340028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:27.429719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:27.435748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dec33-2b16-433d-bdad-a6578c431677 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:27.508440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8cb3c-e448-4b2c-87d1-17351d563f01 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:27.516208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:27.518053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:27.518819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:27.519396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:27.575685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:27.576575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:27.577003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:27.577290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:27.578064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:27.578373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:27.579921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:27.600444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 253/513] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:27.601885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:27.660702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:27.664592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:27.674750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:27.774161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c7f4d-56d1-49b1-826d-013e32155ddc None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:27.841274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:27.842635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:27.843024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:27.843496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:27.858276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:27.861710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:27.864436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:27.869105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 261/514] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:27.871129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:27.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:27.912254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:27.914053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:27.914053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:27.914053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:27.914053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:27.914053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 254/515] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:27.914599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:28.019557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29c88f-8f9a-4eb1-b1a0-9fc0eb2b9ae8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:28.104876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:28.106199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:28.106669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:28.107117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:28.108430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:28.108967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:28.111135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:28.111529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:28.117281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601fbaa-7f1a-4810-bb68-5f00704a089a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:28.140704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 262/516] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:28.140704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:28.190256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:28.191526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:28.192553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:28.193058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:28.209044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:28.211692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:28.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:28.241998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:28.243420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 255/517] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:28.245776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:28.269117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:28.356979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0a42b-1476-4d9a-89d6-e8c43a5d400a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:28.437219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:28.439061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:28.442641 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:28.443046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:28.450564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475d427-3e0c-41ee-a170-e07a2c678c3f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:28.465438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:28.465438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:28.508419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:28.510426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:28.512083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:28.512083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:28.512083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:28.512083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:28.512083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 263/518] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:28.513035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:28.538448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:28.539644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:28.540179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:28.540699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:28.542184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:28.543345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:28.545480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:28.569052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 256/519] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:28.570251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:28.612486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:28.700624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba6e4c-b262-4cc6-b6c8-13a72ddc2adf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:28.726605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:28.768149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:28.769474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:28.770114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:28.770577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:28.776234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd886fdd-825b-41af-9065-009b2422d32a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:28.787809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:28.791364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:28.822900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:28.824582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:28.826131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 264/520] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:28.827033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:28.852210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:28.854048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:28.854896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:28.855714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:28.857876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:28.858208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:28.861318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:28.902666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 257/521] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Jun 16 14:41:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:28.904319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:29.013368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620b7e5-9d0e-4e83-b52d-24e8c5496c63 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:29.044631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:29.071695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:29.073234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:29.073775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:29.074243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:29.075831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:29.076996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:29.079818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:29.087348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbad08-800a-4664-91bf-4ebd3599dd51 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:29.105284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 265/522] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:29.106582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:29.166346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:29.167619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:29.168127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:29.168634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:29.170084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:29.182046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:29.183555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:29.199870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:29.201120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 258/523] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:29.202107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:29.280245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdfba2-2a4e-4170-b74b-19e7ac3c155d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:29.351647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:29.352820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:29.353330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:29.353801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:29.355207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:29.355748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:29.358367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:29.393375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 266/524] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:29.394822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:29.435388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde4c34-4279-49e1-bf7f-47d5be12785c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:29.507857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:29.509075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:29.509541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:29.510050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:29.528071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:29.570075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:29.589431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f45d8d-976d-46e8-ad68-6f8ef7fb7f7c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:29.647101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:29.648710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:29.649065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:29.649346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:29.652256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:29.655282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:29.659472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:29.661646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:29.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:29.679238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 267/525] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:29.680977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:29.861547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252472-6c5a-439d-9964-c304d860966c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:29.899320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 259/526] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:29.899320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:29.933915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:29.935375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:29.935921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:29.936414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:29.938417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:29.938941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:29.942598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:29.976453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:30.023251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 268/527] 10.222.0.72 () {56 vars in 1139 bytes} [Thu Jun 16 14:41:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:30.023251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:30.044430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:30.097695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ca63d-2a01-4b4c-b493-bd80c60d042e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:30.145806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:30.146859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:30.147379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:30.147928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:30.149647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0239f0ca3ab3423491d3844fd79a2868) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:30.150651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:30.153070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:30.163559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 260/528] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:29 2022] GET /identity/v3/role_assignments?user.id=0239f0ca3ab3423491d3844fd79a2868&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:30.164473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:30.192412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ec3bc-c867-401e-ae91-75dbcdd0221f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:30.260450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:30.261200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:30.261519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:30.261859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:30.263002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:30.263333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:30.265688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:30.290051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 269/529] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:30.290051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:30.365416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe84217-ec38-4555-b262-73687ed1f25a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:30.367804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:30.408767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:30.420137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:30.421749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:30.422377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:30.422845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:30.439256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:30.441774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:30.469090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:30.470143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:30.470976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:30.470976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:30.470976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:30.470976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:30.474730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 261/530] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:30.476350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:30.484738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b864316-30e8-40d3-a982-8b8493934801 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:30.561578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5bb03-bcca-46e3-bf07-b9f12bb4d47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:30.563227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5bb03-bcca-46e3-bf07-b9f12bb4d47c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:30.564167 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5bb03-bcca-46e3-bf07-b9f12bb4d47c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:30.565160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5bb03-bcca-46e3-bf07-b9f12bb4d47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:30.616827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb8b9b-eb42-47cf-b0cb-c7db85e342ac None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:30.660134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:30.661240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:30.661240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:30.661492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:30.662256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0b8661ae3402445c9c027e739983f26e) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:30.662583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:30.664201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:30.677334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 262/531] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:30 2022] GET /identity/v3/role_assignments?user.id=0b8661ae3402445c9c027e739983f26e&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:30.678681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:30.730286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:30.735251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5bb03-bcca-46e3-bf07-b9f12bb4d47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:30.741825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5bb03-bcca-46e3-bf07-b9f12bb4d47c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:30.929064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:30.986567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b9535-edd0-4f5a-9976-ed05b9ecf6be None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:30.997908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 270/532] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:30.999987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe43c2-e764-434d-a082-c61a58af641f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:31.000447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe43c2-e764-434d-a082-c61a58af641f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:31.001029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe43c2-e764-434d-a082-c61a58af641f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:31.010979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 271/533] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:31.010979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe43c2-e764-434d-a082-c61a58af641f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:31.065955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10bf75-58ce-4cc5-b8af-af8b36989f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:31.068614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10bf75-58ce-4cc5-b8af-af8b36989f34 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:31.069505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10bf75-58ce-4cc5-b8af-af8b36989f34 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:31.070252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10bf75-58ce-4cc5-b8af-af8b36989f34 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:31.071832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10bf75-58ce-4cc5-b8af-af8b36989f34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:31.072716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10bf75-58ce-4cc5-b8af-af8b36989f34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:31.075699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10bf75-58ce-4cc5-b8af-af8b36989f34 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:31.101896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 263/534] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:31.103325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1c454-1f48-448e-b72a-004518713fb3 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:31.103606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1c454-1f48-448e-b72a-004518713fb3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:31.103888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1c454-1f48-448e-b72a-004518713fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:31.129915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-cff1c454-1f48-448e-b72a-004518713fb3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:31.131772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cff1c454-1f48-448e-b72a-004518713fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:31.213190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe43c2-e764-434d-a082-c61a58af641f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:31.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 264/535] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:31.241532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1c454-1f48-448e-b72a-004518713fb3 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:31.257480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:31.295730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:31.296827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:31.297153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:31.297432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:31.311699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:31.313454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:31.346906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:31.349042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:31.351580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:31.351580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:31.351580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:31.351580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:31.351580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 272/536] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:31.352661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:31.419737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:31.421508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1c454-1f48-448e-b72a-004518713fb3 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:31.497307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:31.500300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:31.501110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:31.502040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:31.519443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:31.523744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:31.559212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:31.561180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:31.563187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:31.564095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4886d3-86fe-419e-a850-b0ed60e7e347 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:31.566765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 265/537] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:31.568552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:31.640440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:31.643284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:31.644045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:31.644701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:31.673748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:31.681405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:31.692419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:31.694223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 273/538] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:31.695769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:31.742265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cc9f3-3ba8-4a61-851f-35b1aafde4b0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:31.800307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:31.801289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:31.801720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:31.802019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:31.803253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:31.804510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:31.806762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:31.836635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 266/539] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:31.838606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:31.897573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:31.931629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a0679d-1bef-4247-83ae-097ef70fa759 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:31.976745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9e5d-49b5-4396-8356-c4c85ae1875e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:32.004222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:32.005628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:32.006132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:32.007070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:32.008405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:32.009017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:32.011692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:32.014384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:32.023584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:32.025257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:32.025895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:32.026389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:32.027383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:32.027910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:32.029590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:32.046501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 274/540] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:32.046501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:32.046780 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 267/541] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:32.048555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:32.206096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a339b1-a7da-448a-9f6c-1895d5bdaa58 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:32.257404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d20ea-b799-4509-8225-8ac6e2fd0c16 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:32.274882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:32.276613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:32.277473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:32.278364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:32.280395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=6812dddc3db746b6893f12f1ae0c715d) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:32.281445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:32.284511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:32.302392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 268/542] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:32 2022] GET /identity/v3/role_assignments?user.id=6812dddc3db746b6893f12f1ae0c715d&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:32.304540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:32.331467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:32.341041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:32.341041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:32.341041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:32.370219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:32.376175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:32.433470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:32.435191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:32.436812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:32.438555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 275/543] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:32.451637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:32.456240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:32.538919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:32.664017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f089f7c-cc9a-4e5b-a1fe-c2cbfaa50fd2 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:32.738183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:32.739585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:32.739706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:32.740063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:32.761983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:32.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:32.778944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f3f23-0e1a-452f-8275-a72455b3ad1a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:32.803145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:32.804767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:32.806413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:32.806413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:32.806413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:32.806413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:32.806413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 269/544] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:32.821180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:32.887903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:32.889554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:32.890388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:32.891161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:32.893380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:32.895510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:32.899699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:32.931101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 276/545] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:32.940109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:33.141396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:33.167336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93111ee-d6cf-4388-8e28-f662ef544332 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:33.181292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:33.255998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded902a-ee09-4164-b4ea-a3e180b75ec4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:33.285219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:33.286447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:33.286963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:33.287465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:33.288935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:33.289511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:33.292233 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:33.318420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 270/546] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:33.326318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:33.388462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7fcf6-0ae0-41ad-8a6e-66037196e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:33.390904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7fcf6-0ae0-41ad-8a6e-66037196e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:33.391907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7fcf6-0ae0-41ad-8a6e-66037196e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:33.392925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7fcf6-0ae0-41ad-8a6e-66037196e6b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:33.394841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7fcf6-0ae0-41ad-8a6e-66037196e6b4 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=2fc5135327fe41409cc4ba407b004d1a) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:33.396123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7fcf6-0ae0-41ad-8a6e-66037196e6b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:33.400255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7fcf6-0ae0-41ad-8a6e-66037196e6b4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:33.427757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 277/547] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:32 2022] GET /identity/v3/role_assignments?user.id=2fc5135327fe41409cc4ba407b004d1a&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 259 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:33.451365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118407a-c8f5-4a2c-b9b3-2a2a7661af77 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:33.452282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118407a-c8f5-4a2c-b9b3-2a2a7661af77 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:33.453051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118407a-c8f5-4a2c-b9b3-2a2a7661af77 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:33.456090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 278/548] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:41:33.471140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9cdb9-67f6-4f41-b858-1353c1edfde4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:33.472203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9cdb9-67f6-4f41-b858-1353c1edfde4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:33.472989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9cdb9-67f6-4f41-b858-1353c1edfde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:33.534813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f3d9cdb9-67f6-4f41-b858-1353c1edfde4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:33.537699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f3d9cdb9-67f6-4f41-b858-1353c1edfde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:33.648740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de70c0-3be9-4f93-80b0-3f37e904c36a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:33.724314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:33.726113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:33.726935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:33.727687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:33.745392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:33.749653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:33.761196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 279/549] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:33.785478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:33.787464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:41:33.789404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:33.794223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 271/550] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:41:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:33.808116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9cdb9-67f6-4f41-b858-1353c1edfde4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:33.829826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:33.919419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:33.957472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:34.048102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9cdb9-67f6-4f41-b858-1353c1edfde4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:34.109671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:34.110431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:34.110716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:34.110995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:34.111926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:34.116000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:34.117865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:34.142421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 280/551] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Jun 16 14:41:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:34.150902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:34.216720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ab67f-1434-4c3f-a046-c96853a75036 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:34.321341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb67f0-6028-4b05-8d3f-fca387e3e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:34.322986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb67f0-6028-4b05-8d3f-fca387e3e67d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:34.323630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb67f0-6028-4b05-8d3f-fca387e3e67d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:34.324103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb67f0-6028-4b05-8d3f-fca387e3e67d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:34.350994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb67f0-6028-4b05-8d3f-fca387e3e67d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4acb67f0-6028-4b05-8d3f-fca387e3e67d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:34.366799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:34.368608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:34.370348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:34.372964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 272/552] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 541 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:34.375509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e457b83-7be5-444d-a8fd-3994c7008466 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:34.375995 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e457b83-7be5-444d-a8fd-3994c7008466 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:34.376477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e457b83-7be5-444d-a8fd-3994c7008466 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:34.385455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 273/553] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:34.386840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e457b83-7be5-444d-a8fd-3994c7008466 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:34.413714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d19af5-9fcb-487e-891c-2836b303897f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:34.441835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:34.480710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:34.481872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:34.482462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:34.483048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:34.562639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:34.596101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e457b83-7be5-444d-a8fd-3994c7008466 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:34.613056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:34.616505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:34.667318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1970ac-e7bf-454e-a551-17bc34110f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:34.669201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1970ac-e7bf-454e-a551-17bc34110f6b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:34.669957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1970ac-e7bf-454e-a551-17bc34110f6b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:34.670824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1970ac-e7bf-454e-a551-17bc34110f6b None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:34.672567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1970ac-e7bf-454e-a551-17bc34110f6b None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:34.673447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1970ac-e7bf-454e-a551-17bc34110f6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:34.676581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1970ac-e7bf-454e-a551-17bc34110f6b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:34.699571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 274/554] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:34.702851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff0e9f-ae5e-41c9-9501-5adeb4f7f55c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:34.703836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff0e9f-ae5e-41c9-9501-5adeb4f7f55c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:34.704655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff0e9f-ae5e-41c9-9501-5adeb4f7f55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:34.737930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-74ff0e9f-ae5e-41c9-9501-5adeb4f7f55c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:34.740992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-74ff0e9f-ae5e-41c9-9501-5adeb4f7f55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:34.817530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 281/555] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:34.819134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:34.846214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:34.878675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 275/556] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:34.881321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff0e9f-ae5e-41c9-9501-5adeb4f7f55c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:35.018767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be08fb-a461-40eb-99fd-f51e74e7a6d1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:35.036694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:35.085530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff0e9f-ae5e-41c9-9501-5adeb4f7f55c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:35.088684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:35.090001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:35.090578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:35.091112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:35.104287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:35.106438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:35.143383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:35.145388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:35.146660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:35.146660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:35.146660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:35.146660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:35.157338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 282/557] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:35.157910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:35.159454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:35.159824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:35.159996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:35.161069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:35.183619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:35.186717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:35.224655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:35.226133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:35.227432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:35.227432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:35.227432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:35.227432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:35.227432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 276/558] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:35.228643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:35.297921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:35.369527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0758226-b215-4c37-b642-949c7c971b24 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:35.444351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:35.447131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:35.447959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:35.448306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada59cae-94fd-4669-876d-7c4629bcf215 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:35.449045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:35.474709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:35.490948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:35.492917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:35.494796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:35.500190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 283/559] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:35.502514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:35.516011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:35.528156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:35.529769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:35.530458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:35.530971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:35.532503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:35.533089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:35.536496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:35.569659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 277/560] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:35.571066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:35.740867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54474965-83b3-428d-aa5e-797722ae6fdc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:35.783001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:35.790372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f906b-3f02-4833-b381-0ca20a8fd3a9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:35.797444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:35.798328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:35.798750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:35.799230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:35.800282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:35.800826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:35.803298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:35.825600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 284/561] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:35.828546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:35.857673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca3429-6e11-4675-b896-e27f7ddee26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:35.859154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca3429-6e11-4675-b896-e27f7ddee26e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:35.859825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca3429-6e11-4675-b896-e27f7ddee26e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:35.860457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca3429-6e11-4675-b896-e27f7ddee26e None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:35.862123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca3429-6e11-4675-b896-e27f7ddee26e None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:35.862659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca3429-6e11-4675-b896-e27f7ddee26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:35.865209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca3429-6e11-4675-b896-e27f7ddee26e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:35.885840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 278/562] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:35.885840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aeb64-56b9-4c5b-9424-c6b1e83aefe7 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:35.885840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aeb64-56b9-4c5b-9424-c6b1e83aefe7 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:35.886816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aeb64-56b9-4c5b-9424-c6b1e83aefe7 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:35.888759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 279/563] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:35.890204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aeb64-56b9-4c5b-9424-c6b1e83aefe7 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:35.985133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:36.039703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d34028-20c1-4835-a7db-32581d50c70a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:36.083596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aeb64-56b9-4c5b-9424-c6b1e83aefe7 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:36.090527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:36.091547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:36.091965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:36.092379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] PATH_INFO: `/v3/users/glance` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:36.103478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:36.105401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:36.126838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:36.128745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 285/564] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:36.134294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:36.151379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da59e0b-39da-432e-a70f-956ea30ede67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:36.152532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da59e0b-39da-432e-a70f-956ea30ede67 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:36.153044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da59e0b-39da-432e-a70f-956ea30ede67 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:36.153496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da59e0b-39da-432e-a70f-956ea30ede67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:36.164640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da59e0b-39da-432e-a70f-956ea30ede67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:36.166911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da59e0b-39da-432e-a70f-956ea30ede67 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-1da59e0b-39da-432e-a70f-956ea30ede67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:36.193061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:36.194801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:36.196438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:36.200327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 280/565] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:36.202810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74a9cc-4652-466b-b7b4-190deea229ca None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:36.203313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74a9cc-4652-466b-b7b4-190deea229ca None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:36.203775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74a9cc-4652-466b-b7b4-190deea229ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:36.235719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0b74a9cc-4652-466b-b7b4-190deea229ca None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:36.236633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:36.240659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0b74a9cc-4652-466b-b7b4-190deea229ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:36.382051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161438c-b57d-4d5c-8355-fbb089f3831e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:36.407032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 281/566] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:36.408297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74a9cc-4652-466b-b7b4-190deea229ca None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:36.468879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:36.470137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:36.470570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:36.471031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:36.495217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:36.498087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:36.519418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:36.531713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:36.533542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:36.536072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:36.536072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:36.536072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:36.536072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:36.542300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 286/567] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:36.544002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:36.659123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74a9cc-4652-466b-b7b4-190deea229ca None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:36.721510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:36.722786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:36.723334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:36.723822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:36.725198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:36.725778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:36.727576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:36.750964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 282/568] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Jun 16 14:41:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:36.753017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:36.772098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:36.782642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475e246-1eba-4fb3-8a3b-423a2811813a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:36.835143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:36.837807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:36.838781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:36.839509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:36.841723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:36.843242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:36.847015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:36.880274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 287/569] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:36.883567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:36.926003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:36.932198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f011602-54b4-4360-a0ce-3baaba9bf6de None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:36.989807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:36.990659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:36.990942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:36.991227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.013582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:37.029694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:37.030761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:37.031799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:37.032498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 283/570] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:37.033624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:37.085859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485913d-77a3-42d7-9377-1bba68e42190 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:37.149109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:37.150256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.150749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.151240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.152606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:37.153189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:37.155849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:37.180979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 288/571] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:37.181808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:37.196158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419fd28-7542-40a8-ad75-32b153867be2 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:37.218206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:37.257642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eed869-9fda-443d-ac3b-4eae738b6596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:37.259095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eed869-9fda-443d-ac3b-4eae738b6596 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.259600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eed869-9fda-443d-ac3b-4eae738b6596 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.260093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eed869-9fda-443d-ac3b-4eae738b6596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.282183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eed869-9fda-443d-ac3b-4eae738b6596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:37.285195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eed869-9fda-443d-ac3b-4eae738b6596 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-e6eed869-9fda-443d-ac3b-4eae738b6596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:37.325331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:37.327107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:37.328924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:37.328924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:37.328924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:37.328924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:37.328924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 284/572] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:37.330037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681856-3b35-4e5a-8b66-f8f637af4419 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.330553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681856-3b35-4e5a-8b66-f8f637af4419 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.331034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681856-3b35-4e5a-8b66-f8f637af4419 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.336047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 285/573] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:37.336324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba681856-3b35-4e5a-8b66-f8f637af4419 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:37.365354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:37.381193 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738299a-789b-496c-a10d-459d89ba969f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:37.447802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:37.448525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.448953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.449319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.450519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=2013d4199e3841109b7c4661ff45b160) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:37.450926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:37.452828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:37.460262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 289/574] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:37 2022] GET /identity/v3/role_assignments?user.id=2013d4199e3841109b7c4661ff45b160&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:37.461008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:37.540973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba681856-3b35-4e5a-8b66-f8f637af4419 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:37.603713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:37.605228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.606368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.606893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.608811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:37.609400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:37.612074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:37.641075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 286/575] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:37.641075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:37.648184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:37.688763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46920e-849b-4a49-a9af-4a92b627432d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:37.756053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d2d23-4ff8-4b17-af4c-758b3d5c07f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:37.758130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2d23-4ff8-4b17-af4c-758b3d5c07f3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.763552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2d23-4ff8-4b17-af4c-758b3d5c07f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.764317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2d23-4ff8-4b17-af4c-758b3d5c07f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.793869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d2d23-4ff8-4b17-af4c-758b3d5c07f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:37.796691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d2d23-4ff8-4b17-af4c-758b3d5c07f3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bd5d2d23-4ff8-4b17-af4c-758b3d5c07f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:37.825662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:37.827512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:37.829975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 290/576] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:41:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:37.830875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58fc9-f12b-434e-96be-b72f1c5ce943 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.831519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58fc9-f12b-434e-96be-b72f1c5ce943 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.831789 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b58fc9-f12b-434e-96be-b72f1c5ce943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.844105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:37.863156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-71b58fc9-f12b-434e-96be-b72f1c5ce943 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:37.865490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-71b58fc9-f12b-434e-96be-b72f1c5ce943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:37.901982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b58a5-7030-464e-a48b-a16a96727def None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:37.977145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:37.978333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:37.978933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:37.979477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:37.980944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:37.981488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:37.984127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:38.014840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 287/577] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:38.015799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:38.039894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 291/578] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:38.042035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b58fc9-f12b-434e-96be-b72f1c5ce943 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:38.183409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:38.227804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b58fc9-f12b-434e-96be-b72f1c5ce943 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:38.233808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b216d9-e79e-4cc9-9e6f-28984b2a3ea0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:38.286858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:38.288468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:38.289006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:38.290194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:38.291817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:38.292292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:38.294930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:38.312460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:38.314086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:38.314655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:38.315196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:38.319996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:38.321306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 292/579] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:38.327124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:38.340830 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:38.342698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:38.382946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:38.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:38.386877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:38.386877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:38.386877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:38.386877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:38.386877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 288/580] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:38.386877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:38.495858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774d1f3-059f-40f3-927b-f3351e44f07d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:38.555183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:38.564597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:38.565506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:38.565796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:38.566298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:38.581456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:38.584601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:38.599115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1483a-c446-4489-8bad-e9ac484b9993 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:38.605195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:38.606514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 293/581] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:38.613092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:38.662405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:38.663267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:38.663555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:38.663838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:38.680362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:38.688508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:38.689586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:38.690426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 289/582] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:38.691427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:38.773379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:38.812184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bfdad-f548-44c0-a5ec-fb57494d2aa6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:38.877120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:38.878472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215fe92-b95a-44f8-b895-c694f9ba5388 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:38.883415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:38.883941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:38.884459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:38.905250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:38.907452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:38.931084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:38.933007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:38.934793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:38.934793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:38.934793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:38.934793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:38.934793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 294/583] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:38.935346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:38.966768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:38.967843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:38.968287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:38.968939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:38.969930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:38.970613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:38.972305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:38.995652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 290/584] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:38.996957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:39.037152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:39.169127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace27868-495b-4809-a418-4d26fe082ede None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:39.181181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb09c22-c2c7-41a1-851f-a342102e951e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:39.197147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:39.246709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:39.248015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:39.248561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:39.249136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:39.250887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:39.251502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:39.255181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:39.259403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:39.261228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:39.262127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:39.262957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:39.264577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:39.265567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:39.269159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:39.282720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 295/585] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:39.284409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:39.295277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 291/586] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:39.297621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:39.486815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf74de7-bd13-4cdd-a006-57eeefedeff1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:39.493471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:39.532199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94963f9-3217-4354-8e2b-bb925dba6e65 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:39.566059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:39.567420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:39.567718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:39.567999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:39.569122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:39.569620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:39.571647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:39.600894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 296/587] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:39.600894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:39.614878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:39.616480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:39.617080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:39.617626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:39.640029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:39.643443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:39.673368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:39.675376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:39.677918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 292/588] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:39.685653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:39.743384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:39.846325 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eaee8-00b7-4fba-95b7-6198f9ba0426 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:39.901381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85ee1b-ae2c-4277-a3db-0985063110fc None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:39.917125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:39.918938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:39.919458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:39.920132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0239f0ca3ab3423491d3844fd79a2868/roles/5a44c123efd54f14bc8e76881f748f8b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:39.958637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=0239f0ca3ab3423491d3844fd79a2868, role_id=5a44c123efd54f14bc8e76881f748f8b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:39.961516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:39.972351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:39.973815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:39.974308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:39.974773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:39.996983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 297/589] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:41:39 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0239f0ca3ab3423491d3844fd79a2868/roles/5a44c123efd54f14bc8e76881f748f8b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:39.996983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:40.008517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:40.011806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:40.031884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:40.055327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:40.056356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:40.057480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:40.057480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:40.057480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:40.057480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:40.065436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 293/590] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:40.067807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:40.244806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41d9a9-a37f-424e-828c-b4479baaa95a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:40.253383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:40.301323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0336d8-e856-470c-aee8-0e78db4d1ebd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:40.326149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:40.333938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:40.334431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:40.334880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:40.352988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:40.356161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:40.369166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:40.370426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:40.370921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:40.371414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:40.373005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:40.373558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:40.376341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:40.391322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:40.393437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:40.395181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:40.401184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 298/591] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:40.405430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:40.414192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 294/592] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Jun 16 14:41:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:40.421386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:40.584227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:40.631607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68125c-24b5-46b4-9ba5-bdd573cb7ca1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:40.648328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c72b3-06ee-4ccb-851b-73902e5795ed None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:40.705069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:40.706610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:40.707922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:40.708692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:40.710761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:40.711639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:40.714717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:40.718735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:40.719851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:40.720325 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:40.720794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:40.722149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:40.722740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:40.726009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:40.739451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:40.760223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 299/593] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:40.761359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 295/594] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:40.762482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:40.763121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:40.977182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfd61b-c5e6-4479-ad6e-aa9b59507f67 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:40.989593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac4aab-6106-449b-a289-87b269672fa7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:41.039158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:41.048116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da670-393e-4864-abd4-94e66d578136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:41.049744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da670-393e-4864-abd4-94e66d578136 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.050337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da670-393e-4864-abd4-94e66d578136 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.051375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da670-393e-4864-abd4-94e66d578136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.064694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:41.067687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.068425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.069393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.072907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184da670-393e-4864-abd4-94e66d578136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:41.075519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184da670-393e-4864-abd4-94e66d578136 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:41.086122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:41.089641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-184da670-393e-4864-abd4-94e66d578136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:41.108264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:41.110061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:41.111769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:41.111769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:41.111769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:41.111769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:41.117379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 300/595] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:41.118993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da670-393e-4864-abd4-94e66d578136 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:41.127343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:41.129181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:41:41.130991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:41.135739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 296/596] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:41.143035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:41.266729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:41.274656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da670-393e-4864-abd4-94e66d578136 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:41.325552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:41.326571 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.326918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.327259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0b8661ae3402445c9c027e739983f26e/roles/5a44c123efd54f14bc8e76881f748f8b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.353106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e40c-6ced-44b6-94b2-5558d285eeda None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:41.353543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=0b8661ae3402445c9c027e739983f26e, role_id=5a44c123efd54f14bc8e76881f748f8b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:41.356054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:41.384883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 301/597] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:41:41 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0b8661ae3402445c9c027e739983f26e/roles/5a44c123efd54f14bc8e76881f748f8b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:41.384883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:41.428977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d347922-cd34-46f3-8394-51ca0bc76235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:41.430342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d347922-cd34-46f3-8394-51ca0bc76235 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.430722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d347922-cd34-46f3-8394-51ca0bc76235 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.431210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d347922-cd34-46f3-8394-51ca0bc76235 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.432550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d347922-cd34-46f3-8394-51ca0bc76235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:41.433034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d347922-cd34-46f3-8394-51ca0bc76235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:41.435685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d347922-cd34-46f3-8394-51ca0bc76235 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:41.455202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:41.466027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 297/598] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:41.469160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5beb-6a20-40b0-8746-cb23015f7ba1 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.469854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5beb-6a20-40b0-8746-cb23015f7ba1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.470511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e5beb-6a20-40b0-8746-cb23015f7ba1 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.477484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 298/599] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:41.479375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5beb-6a20-40b0-8746-cb23015f7ba1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:41.638105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6018a-8fcb-4ff4-8cbd-0ca8687a367b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:41.726056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b1a21-6f84-47f1-8fab-61191c2360b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:41.727313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1a21-6f84-47f1-8fab-61191c2360b1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.727731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1a21-6f84-47f1-8fab-61191c2360b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.728445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1a21-6f84-47f1-8fab-61191c2360b1 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.730801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b1a21-6f84-47f1-8fab-61191c2360b1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:41.730801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b1a21-6f84-47f1-8fab-61191c2360b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:41.735291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b1a21-6f84-47f1-8fab-61191c2360b1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:41.739432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e5beb-6a20-40b0-8746-cb23015f7ba1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:41.776876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 302/600] 10.222.0.72 () {56 vars in 1139 bytes} [Thu Jun 16 14:41:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:41.776876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b6a2f-18fc-4c60-880c-545aa5dba3eb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.776876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b6a2f-18fc-4c60-880c-545aa5dba3eb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.777368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b6a2f-18fc-4c60-880c-545aa5dba3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.784963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:41.821203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-846b6a2f-18fc-4c60-880c-545aa5dba3eb None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:41.825194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-846b6a2f-18fc-4c60-880c-545aa5dba3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:41.842430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:41.845299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:41.845997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:41.846553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:41.861741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:41.865078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:41.900162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:41.901813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:41.902781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 299/601] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:41.903286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:41.994384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 303/602] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:41.996037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b6a2f-18fc-4c60-880c-545aa5dba3eb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:42.046958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:42.138976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863483d0-193a-4579-8b30-64bd701a8afd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:42.196550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b6a2f-18fc-4c60-880c-545aa5dba3eb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:42.216320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:42.217748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:42.218234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:42.218712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:42.241768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:42.244953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:42.267977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:42.270233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:42.270983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:42.271675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:42.273360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:42.274186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:42.277014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:42.283521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:42.285900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:42.287631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:42.287631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:42.287631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:42.287631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:42.293529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 300/603] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:42.295142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:42.307965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 304/604] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:42.309822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:42.360912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:42.475188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:42.510164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bb25f-2ead-4cea-b622-f45c5c2ccf20 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:42.510407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7679e-e030-4fae-a3ae-e3a0e7865d4f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:42.578448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:42.579539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:42.580014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:42.580484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:42.580754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:42.581847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:42.582442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:42.582962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:42.583664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:42.584371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:42.584981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:42.605961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:42.615108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 305/605] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:42.615108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:42.627361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:42.629292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:42.630913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:42.636978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 301/606] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:42.639299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:42.815925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f7375-049b-45ae-9be8-e5c8f8310293 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:42.837000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:42.859387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d684c0-8571-4afb-aec0-84c9d089f016 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:42.891689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:42.893936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:42.894588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:42.896237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/6812dddc3db746b6893f12f1ae0c715d/roles/5a44c123efd54f14bc8e76881f748f8b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:42.924633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:42.931648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=6812dddc3db746b6893f12f1ae0c715d, role_id=5a44c123efd54f14bc8e76881f748f8b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:42.935392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:42.946551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:42.947922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:42.948419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:42.949380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:42.965139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:42.965587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 306/607] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:41:42 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/6812dddc3db746b6893f12f1ae0c715d/roles/5a44c123efd54f14bc8e76881f748f8b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:42.967383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:42.969349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:42.999408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:43.001055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:43.002928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:43.008416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 302/608] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:43.017904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:43.303113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafda1e1-97b4-4592-9156-7b7532dadde3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:43.377625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9c98d-42b8-4131-b8dc-2cdfdb2329f3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:43.401583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:43.403317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:43.404171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:43.405022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:43.408002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:43.409050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:43.414184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:43.454678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 303/609] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:43.464805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:43.480473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:43.481280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:43.481588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:43.482058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:43.483057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:43.483420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:43.485019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:43.493384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:43.513162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 307/610] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:43.523709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:43.689546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:43.757254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747101c3-da77-4049-830f-9ad66ec4d178 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:43.793165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a743af6-703b-4860-9445-ab1e7a2ab30d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:43.828895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50635571-8fc0-45e0-8262-86d31b2aaaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:43.829925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50635571-8fc0-45e0-8262-86d31b2aaaab None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:43.830220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50635571-8fc0-45e0-8262-86d31b2aaaab None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:43.830615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50635571-8fc0-45e0-8262-86d31b2aaaab None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/2fc5135327fe41409cc4ba407b004d1a/roles/5a44c123efd54f14bc8e76881f748f8b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:43.844176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7027e2-f9dd-4e68-8123-af1841ea474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:43.845141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7027e2-f9dd-4e68-8123-af1841ea474a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:43.845462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7027e2-f9dd-4e68-8123-af1841ea474a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:43.845702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7027e2-f9dd-4e68-8123-af1841ea474a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:43.858596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50635571-8fc0-45e0-8262-86d31b2aaaab None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=2fc5135327fe41409cc4ba407b004d1a, role_id=5a44c123efd54f14bc8e76881f748f8b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:43.861585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7027e2-f9dd-4e68-8123-af1841ea474a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:43.862717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50635571-8fc0-45e0-8262-86d31b2aaaab None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:43.863619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7027e2-f9dd-4e68-8123-af1841ea474a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0a7027e2-f9dd-4e68-8123-af1841ea474a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:43.896916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:43.898984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:43.901236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:43.901236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:43.901236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:43.901236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:43.903447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 308/611] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:43.905048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a953ebd-ca36-4e67-b2f9-1b5b5b876596 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:43.905337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a953ebd-ca36-4e67-b2f9-1b5b5b876596 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:43.905611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a953ebd-ca36-4e67-b2f9-1b5b5b876596 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:43.907369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 309/612] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:43.908306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a953ebd-ca36-4e67-b2f9-1b5b5b876596 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:43.908717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 304/613] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:41:43 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/2fc5135327fe41409cc4ba407b004d1a/roles/5a44c123efd54f14bc8e76881f748f8b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:43.916896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2418067-29c0-4d56-9a68-32d0fe215822 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:43.920876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2418067-29c0-4d56-9a68-32d0fe215822 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:43.920876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2418067-29c0-4d56-9a68-32d0fe215822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:43.970371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f2418067-29c0-4d56-9a68-32d0fe215822 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:43.971699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f2418067-29c0-4d56-9a68-32d0fe215822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:44.104886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a953ebd-ca36-4e67-b2f9-1b5b5b876596 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:44.111794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:44.146858 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:44.179500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 305/614] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:44.181196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:44.182735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:44.183552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:44.184274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:44.185946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:44.186771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:44.189713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:44.220733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 310/615] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:44.230842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2418067-29c0-4d56-9a68-32d0fe215822 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:44.249797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:44.460284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2418067-29c0-4d56-9a68-32d0fe215822 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:44.553936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:44.555412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:44.555963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:44.556486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] PATH_INFO: `/v3/users/swift` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:44.560574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7ab7a-d3bb-4187-9f29-18cb6d4d1dd6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:44.590925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:44.593910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:44.621918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:44.623373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:44.623919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:44.624458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:44.640431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:44.644551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:44.648330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 306/616] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:41:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:44.663599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:44.671753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:44.688309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:44.690458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:44.692236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:44.697824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 311/617] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:44.706547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:44.813224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:44.813468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:44.970332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62abf2-2636-4c5a-bfff-38d7b9bfdf7d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:45.034370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5c3d7-62d9-4d52-803b-76b93c06c93d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:45.047557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a7821-f9de-4ad3-9dbf-3e41e247834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:45.049274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7821-f9de-4ad3-9dbf-3e41e247834c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:45.050025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7821-f9de-4ad3-9dbf-3e41e247834c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:45.051377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7821-f9de-4ad3-9dbf-3e41e247834c None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:45.053335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a7821-f9de-4ad3-9dbf-3e41e247834c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:45.054421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a7821-f9de-4ad3-9dbf-3e41e247834c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:45.057534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a7821-f9de-4ad3-9dbf-3e41e247834c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:45.101337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 307/618] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Jun 16 14:41:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:45.104730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da58ec-35ec-432f-975b-2d5985853eb0 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:45.105594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da58ec-35ec-432f-975b-2d5985853eb0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:45.106475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da58ec-35ec-432f-975b-2d5985853eb0 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:45.107377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445243f9-c937-4836-8ddd-915a032420e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:45.108788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445243f9-c937-4836-8ddd-915a032420e3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:45.109482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445243f9-c937-4836-8ddd-915a032420e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:45.110237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445243f9-c937-4836-8ddd-915a032420e3 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:45.110779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 308/619] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:45.114906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da58ec-35ec-432f-975b-2d5985853eb0 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:45.118719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445243f9-c937-4836-8ddd-915a032420e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:45.119426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445243f9-c937-4836-8ddd-915a032420e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:45.123543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445243f9-c937-4836-8ddd-915a032420e3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:45.141693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 312/620] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:45.145143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e8e3d-fc54-478b-9a3e-e250f0130a18 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:45.146003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e8e3d-fc54-478b-9a3e-e250f0130a18 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:45.146689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e8e3d-fc54-478b-9a3e-e250f0130a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:45.180158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e88e8e3d-fc54-478b-9a3e-e250f0130a18 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:45.183006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e88e8e3d-fc54-478b-9a3e-e250f0130a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:45.326864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da58ec-35ec-432f-975b-2d5985853eb0 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:45.328732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 313/621] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:45.329691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e8e3d-fc54-478b-9a3e-e250f0130a18 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:45.347851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:45.387082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:45.398893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:45.400004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:45.400473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:45.401008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:45.421486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:45.424900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:45.461603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:45.464203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:45.466117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:45.466117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:45.466117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:45.466117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:45.469950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 309/622] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:45.478109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:45.567683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e8e3d-fc54-478b-9a3e-e250f0130a18 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:45.647350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:45.648793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:45.650120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:45.650120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:45.674141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:45.677249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:45.685717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f65a-3be0-454f-8970-4bd86cb9257d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:45.715396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:45.717230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:45.718537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:45.718537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:45.718537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:45.718537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:45.718537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 314/623] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:45.720516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:45.759796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:45.761590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:45.762320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:45.763053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:45.784947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:45.800382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:45.802330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:45.803983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:45.809923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 310/624] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:45.811881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:45.858253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:45.889498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:45.953147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72b8f8-c5de-4e28-a420-116c5eeb4834 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:46.020625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb726b6-a817-4e1f-89bd-d24d3d6c734a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:46.024014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:46.025465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.026948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.026948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:46.028294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:46.029042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:46.031815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:46.063272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 315/625] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:46.063272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:46.103036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:46.104786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.105647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.106416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:46.108150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:46.109085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:46.112408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:46.138502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 311/626] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:46.140474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:46.276202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f93775-c586-463d-afb2-14579acc964a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:46.318965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:46.343356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:46.361979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:46.363506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.364116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.364701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:46.366379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:46.367050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:46.370072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:46.370313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414ed67-4915-471d-828f-36f7e4647c24 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:46.393221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 316/627] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:46.396099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:46.456244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:46.458278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.458775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.459289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:46.473400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:46.476312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:46.506147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:46.507906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 312/628] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:46.509934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:46.598006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5a98f-2f17-4db6-bbb1-10314ae77890 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:46.659065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c921cdb-189a-44bf-a866-fbb576071379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:46.660260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c921cdb-189a-44bf-a866-fbb576071379 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.660704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c921cdb-189a-44bf-a866-fbb576071379 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.661191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c921cdb-189a-44bf-a866-fbb576071379 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:46.677309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c921cdb-189a-44bf-a866-fbb576071379 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:46.680766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c921cdb-189a-44bf-a866-fbb576071379 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-1c921cdb-189a-44bf-a866-fbb576071379 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:46.701710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:46.703623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 317/629] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0f6a0-339e-4e65-9b12-fddfa964ffa2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.711311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0f6a0-339e-4e65-9b12-fddfa964ffa2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.712680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0f6a0-339e-4e65-9b12-fddfa964ffa2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:46.712680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 318/630] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:46.712680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0f6a0-339e-4e65-9b12-fddfa964ffa2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:46.729718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e48b62-7f66-49b6-b14d-d4d53e863fef None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:46.793626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:46.794884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.795261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.795643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:46.798367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:46.815457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:46.821254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:46.824664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:46.856094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:46.857961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:46.869468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:46.869468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:46.869468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:46.869468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:46.869468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 313/631] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:46.869468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:46.924262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0f6a0-339e-4e65-9b12-fddfa964ffa2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:46.996441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b992a733-55af-4475-8efd-ee45dbd7bbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:46.997598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a733-55af-4475-8efd-ee45dbd7bbf5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:46.998111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a733-55af-4475-8efd-ee45dbd7bbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:46.998612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992a733-55af-4475-8efd-ee45dbd7bbf5 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.000197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992a733-55af-4475-8efd-ee45dbd7bbf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:47.000947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992a733-55af-4475-8efd-ee45dbd7bbf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:47.003743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b992a733-55af-4475-8efd-ee45dbd7bbf5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:47.025129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 319/632] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:47.027921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d39db-c656-4d14-8573-f0279a78ae07 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:47.028753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d39db-c656-4d14-8573-f0279a78ae07 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:47.029497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d39db-c656-4d14-8573-f0279a78ae07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.058271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-830d39db-c656-4d14-8573-f0279a78ae07 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:47.060422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-830d39db-c656-4d14-8573-f0279a78ae07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:47.060655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587ed6f-962d-4eda-abe0-68b485e2866e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:47.108055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:47.109134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:47.109709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:47.110205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.111815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:47.112318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:47.115811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:47.149982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 314/633] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Jun 16 14:41:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:47.151277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:47.173003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 320/634] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:47.174714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d4a3-b8bc-4ed9-ac44-e7155e491f5c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:47.174941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d4a3-b8bc-4ed9-ac44-e7155e491f5c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:47.175223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d4a3-b8bc-4ed9-ac44-e7155e491f5c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.177358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 321/635] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:47.178142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d4a3-b8bc-4ed9-ac44-e7155e491f5c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:47.251103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:47.265448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:47.339877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52e4d2-0818-4ee4-90a2-091b0769db19 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:47.374184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d4a3-b8bc-4ed9-ac44-e7155e491f5c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:47.406973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:47.408037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:47.408488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:47.408970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.410752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:47.411267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:47.413535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:47.431703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5e25c-ecd7-4f79-81f4-0a08c4d3e1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:47.432715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5e25c-ecd7-4f79-81f4-0a08c4d3e1e9 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:47.433101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5e25c-ecd7-4f79-81f4-0a08c4d3e1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:47.433451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5e25c-ecd7-4f79-81f4-0a08c4d3e1e9 None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/2013d4199e3841109b7c4661ff45b160/roles/5a44c123efd54f14bc8e76881f748f8b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.434779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 315/636] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:47.436176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:47.455133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5e25c-ecd7-4f79-81f4-0a08c4d3e1e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=2013d4199e3841109b7c4661ff45b160, role_id=5a44c123efd54f14bc8e76881f748f8b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:47.458108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5e25c-ecd7-4f79-81f4-0a08c4d3e1e9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:47.491609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 322/637] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:41:47 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/2013d4199e3841109b7c4661ff45b160/roles/5a44c123efd54f14bc8e76881f748f8b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:41:47.493876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65de25-b4dd-4f1b-9610-396ac25497e9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:47.494351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65de25-b4dd-4f1b-9610-396ac25497e9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:47.494949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65de25-b4dd-4f1b-9610-396ac25497e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.535833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-de65de25-b4dd-4f1b-9610-396ac25497e9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:47.546648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-de65de25-b4dd-4f1b-9610-396ac25497e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:47.668992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb1aa1-99cf-4c50-8915-dde785f74ec5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:47.689912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:47.709869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 323/638] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:47.709869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65de25-b4dd-4f1b-9610-396ac25497e9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:47.759493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:47.760267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:47.760543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:47.760892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:47.762287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:47.922520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:47.926276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:47.935774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65de25-b4dd-4f1b-9610-396ac25497e9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:48.001321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:48.002714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:48.003219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:48.003727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:48.027125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:48.044496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:48.046314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:48.047979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 324/639] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:48.052383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:48.168210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 316/640] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:48.168210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:48.191826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:48.241796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:48.279984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61354d-5e2f-42aa-9f81-e67f72a34d73 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:48.339893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:48.341326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:48.341634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:48.341905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:48.352998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:48.354790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:48.381403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02924fc2-9c50-44f9-b239-e48b97e133af None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:48.385308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:48.386508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:48.387662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:48.393451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 325/641] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:41:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:48.396493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:48.443067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:48.443987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:48.444278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:48.444558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:48.470634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:48.484300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:48.486385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:48.488013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 317/642] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:48.492671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:48.595183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd96e2-02bf-48a5-951e-27498e634f1f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:48.626411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:48.647392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:48.648355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:48.648882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:48.649344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:48.650717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:48.651163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:48.653011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:48.668298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 326/643] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:48.670359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:48.678506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:48.699152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eae8e1-0f5a-47a6-b939-097859de0fe8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:48.775950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:48.777749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:48.778313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:48.778965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:48.801128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:48.802816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:48.835073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:48.837123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:48.847470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:48.847470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:48.847470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:48.847470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:48.847470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 318/644] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:48.847470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:48.855931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee00ab-560d-464a-a5de-50cccf8f8407 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:48.923879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:48.926526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:48.927272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:48.928016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:48.929761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:48.930285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:48.933319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:48.966525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 327/645] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:41:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:48.968026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:49.050434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26840b-45e2-4305-8ca2-7462399ee9c3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:49.087036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:49.096005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:49.123619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:49.124812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:49.125314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:49.125790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:49.127255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:49.127813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:49.131159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:49.154737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 319/646] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:49.154737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:49.175301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168886b-bad5-4fc0-9a61-f06b895f105f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:49.248547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:49.249768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:49.250184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:49.250590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:49.274027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:49.276756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:49.317939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:49.324627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:49.325813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:49.325813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:49.325813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:49.325813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:49.325813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 328/647] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:49.325813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:49.357321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86eb4ec-b7a9-45b6-9884-b5c10a8d42ab None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:49.429561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:49.430902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:49.431510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:49.431877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:49.432718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:49.433134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:49.434592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:49.464296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 320/648] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:49.465776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:49.502993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5d16-1554-4c3d-babb-82175d891d86 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:49.527202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:49.547252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:49.568477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:49.569807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:49.570392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:49.571000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:49.635120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f534-7d6d-40ba-b4fc-511d6c01490d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:49.684216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:49.685717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:49.686237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:49.686744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:49.698527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:49.700260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:49.701621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:49.703490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:49.731457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:49.732769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:49.739478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:49.739478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:49.739478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:49.739478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:49.739478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 321/649] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:49.739478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:49.903694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 329/650] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:49.903694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:49.914698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:49.924233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:49.933343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cd5dc-60ae-4154-b275-9d4eb1843ab9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:50.003861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:50.005216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:50.005613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:50.006144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:50.007678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:50.008348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:50.012954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:50.030375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 322/651] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:50.031546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:50.107161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdeeaa0-40ab-4093-9d18-09bc0021712a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:50.175138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:50.176831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:50.177873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:50.177873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:50.188046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:50.190109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:50.209377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:50.211201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 330/652] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:50.213585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:50.218975 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce24b7-efe8-45b0-acf3-809bef8ef842 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:50.280236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:50.281337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:50.281834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:50.282311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:50.283250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:50.283770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:50.286127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:50.306467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 323/653] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:50.307832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:50.332568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:50.338532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:50.382109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193dce4-0bee-4067-9927-2aee3fbeaf0b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:50.460586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:50.461602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:50.461885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:50.462160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:50.478248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:50.480060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:50.504567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e117-b110-412b-bd81-4e4279dab7f8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:50.508695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:50.510548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:50.512336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:50.512336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:50.512336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:50.512336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:50.512336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 331/654] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:50.512336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:50.589482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:50.590996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:50.591570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:50.592080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:50.608987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:50.611912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:50.641487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:50.643568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:50.646933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 324/655] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:50.653667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:50.741902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06bae2-d27f-448a-9b82-a6f914e84fc9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:50.800870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:50.801536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:50.820623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:50.821782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:50.822497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:50.822993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:50.824499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:50.825085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:50.827871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:50.850364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c92d27-b6da-4186-8013-404401568cb0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:50.851611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 332/656] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:50.852647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:50.919015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:50.920433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:50.920944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:50.921463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:50.935927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:50.938853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:50.968580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:50.970294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 325/657] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:50.973034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:51.045638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bf895-eee3-484d-b6b9-77da1a6595b4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:51.114639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107294b-5b2c-4db2-b451-557bcfdc9bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:51.115332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107294b-5b2c-4db2-b451-557bcfdc9bf9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.115722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107294b-5b2c-4db2-b451-557bcfdc9bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.116086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107294b-5b2c-4db2-b451-557bcfdc9bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.117109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107294b-5b2c-4db2-b451-557bcfdc9bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:51.117417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107294b-5b2c-4db2-b451-557bcfdc9bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:51.118851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107294b-5b2c-4db2-b451-557bcfdc9bf9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:51.133279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 333/658] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:51.134680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6b146-ff16-4ca4-a610-ecdfdd28f1c2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.134987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6b146-ff16-4ca4-a610-ecdfdd28f1c2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.135256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6b146-ff16-4ca4-a610-ecdfdd28f1c2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.136877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 334/659] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:51.137413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f78767-c9e1-4267-80f2-b8e51c5d65ab None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:51.137600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6b146-ff16-4ca4-a610-ecdfdd28f1c2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:51.201618 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:51.203891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:51.204983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.205439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.205861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.207589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:51.208090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:51.209831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:51.213659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:51.248306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 326/660] 10.222.0.72 () {56 vars in 1137 bytes} [Thu Jun 16 14:41:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:51.250404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:51.311268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6b146-ff16-4ca4-a610-ecdfdd28f1c2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:51.374280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:51.375151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.375582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.376021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.377446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0239f0ca3ab3423491d3844fd79a2868) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:51.377966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:51.381211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:51.402144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 335/661] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:51 2022] GET /identity/v3/role_assignments?user.id=0239f0ca3ab3423491d3844fd79a2868&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:51.403747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:51.436416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a675ffe-37a0-42af-9bbb-846657bd46de None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:51.501198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff985e5-c5de-441b-b415-723e8e4d9b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:51.502565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff985e5-c5de-441b-b415-723e8e4d9b6e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.503068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff985e5-c5de-441b-b415-723e8e4d9b6e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.503540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff985e5-c5de-441b-b415-723e8e4d9b6e None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.505954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff985e5-c5de-441b-b415-723e8e4d9b6e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:51.506517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff985e5-c5de-441b-b415-723e8e4d9b6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:51.509414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff985e5-c5de-441b-b415-723e8e4d9b6e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:51.543445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 327/662] 10.222.0.72 () {56 vars in 1139 bytes} [Thu Jun 16 14:41:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:51.545328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b694f-f3ae-45cb-a218-7a5e0ee78903 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.546211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b694f-f3ae-45cb-a218-7a5e0ee78903 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.546630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b694f-f3ae-45cb-a218-7a5e0ee78903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.568324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2f4b694f-f3ae-45cb-a218-7a5e0ee78903 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:51.570249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2f4b694f-f3ae-45cb-a218-7a5e0ee78903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:51.619523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdd3ae-178f-4d70-82d5-38b115b0f0bd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:51.635608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:51.677946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:51.684485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:51.685507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.685767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.687464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.711370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:51.713993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:51.721958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 328/663] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:51.724091 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b694f-f3ae-45cb-a218-7a5e0ee78903 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:51.737848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:51.740079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 336/664] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:51.742655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:51.910192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ee92e-8963-443e-aca0-124b5d4604f3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:51.917401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b694f-f3ae-45cb-a218-7a5e0ee78903 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:51.959770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:51.960440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.960722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.961035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:51.989402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:51.991032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:51.991826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:51.992577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:52.037553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:52.074468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:52.101749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:52.103509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:52.103770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:52.106555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:52.274435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 337/665] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:52.275796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:52.279316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 329/666] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:52.281370 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:52.410104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:52.455141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102964e-d468-49a4-993d-7d94104852d0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:52.462071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:52.499550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b64ccc-d068-4cd3-ab1c-be3c01611b28 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:52.523476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:52.525256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:52.525837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:52.526338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:52.527887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:52.528465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:52.532580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:52.560900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 338/667] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:52.560900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:52.567940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:52.569422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:52.569873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:52.570307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:52.590318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:52.606249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:52.607336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:41:52.608283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:52.610717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 330/668] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:52.610717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:52.741336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc3066-87a5-4bf1-bc51-045dcd170d30 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:52.782001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:52.790507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:52.791389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:52.791705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:52.791996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:52.822005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:52.823900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:52.829016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bcd53-aa78-4d46-a916-ec910a34f7e3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:52.859550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:52.861439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:52.864829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:52.864829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:52.864829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:52.864829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:52.864829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 339/669] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:52.864829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:52.905408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:52.906840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:52.907316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:52.907832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:52.928746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:52.931520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:52.935318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:52.974433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:52.976203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:52.978003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:52.978003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:52.978003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:52.978003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:52.978003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 331/670] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:52.978550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:53.052932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a4bba9-6b4c-43fd-9127-f06402119c0b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:53.116156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:53.117217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:53.117683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:53.118143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:53.119626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0b8661ae3402445c9c027e739983f26e) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:53.120137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:53.122319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:53.144893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 340/671] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:52 2022] GET /identity/v3/role_assignments?user.id=0b8661ae3402445c9c027e739983f26e&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:53.144893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:53.157623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb019418-52fd-4d78-9dd4-959286d57799 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:53.227048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:53.228177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:53.228639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:53.229159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:53.230573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:53.231105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:53.233471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:53.245746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:53.261426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 332/672] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:41:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:53.262975 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:53.359899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0081aeb-1e3e-44dc-b74b-a20fc20168b9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:53.383988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:53.434713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:53.435424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:53.435713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:53.436043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:53.437536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:53.437869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:53.439661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:53.460879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 341/673] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:53.460879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:53.492909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ff33-b7b2-466a-832e-2bf5afb1d198 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:53.578440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:53.579515 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:53.579982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:53.580437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:53.581833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:53.582368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:53.584862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:53.608933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 333/674] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:53.610540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:53.705878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad1f5-f81d-42fd-b099-e5ac1c83804a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:53.723226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:53.782499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:53.783936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:53.784438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:53.785030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:53.806520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:53.809364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:53.850597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc78eb-d0f2-4e22-8d6d-aad38f8f4494 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:53.851538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:53.854161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:53.856360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:53.856360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:53.856360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:53.856360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:53.864952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 342/675] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:53.864952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:53.897581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:53.945527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:53.948032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:53.948762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:53.949507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:53.967756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:53.971386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:54.003092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:54.005118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:54.007231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:54.009301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 334/676] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:54.010868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:54.091533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ee05c-116a-4892-926f-1abdc53ae451 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:54.171911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:54.174607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:54.175220 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:54.176393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:54.191767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:54.193860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:54.220685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:54.222805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:54.225280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:54.232425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 343/677] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:54.233948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:54.266843 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:54.270058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b17c58-2c25-4d4a-aed1-a42377105154 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:54.339015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:54.340022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:54.340480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:54.341028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:54.342380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:54.342697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:54.344392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:54.380154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 335/678] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:54.381747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:54.437956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68f163-1d7e-4f8c-9062-1cb64ee95abd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:54.458312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:54.514660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:54.515763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:54.516246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:54.516723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:54.518525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:54.519063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:54.521858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:54.550302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 344/679] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:54.550302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:54.604434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77bb29-4cf5-45b0-a5a0-132b8f95fbab None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:54.655628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:54.675205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53467625-ea32-4339-8f0f-327cd840ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:54.676352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467625-ea32-4339-8f0f-327cd840ad48 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:54.676817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467625-ea32-4339-8f0f-327cd840ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:54.677424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467625-ea32-4339-8f0f-327cd840ad48 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:54.678930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53467625-ea32-4339-8f0f-327cd840ad48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:54.679117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53467625-ea32-4339-8f0f-327cd840ad48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:54.681301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53467625-ea32-4339-8f0f-327cd840ad48 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:54.695680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830b106-3db9-4795-9edd-358c0790e1e2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:54.707576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 336/680] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:54.707576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37be9b3-1d27-4632-87d8-a7fb70a90e2d None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:54.707576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37be9b3-1d27-4632-87d8-a7fb70a90e2d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:54.707576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37be9b3-1d27-4632-87d8-a7fb70a90e2d None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:54.708049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 337/681] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:54.713111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37be9b3-1d27-4632-87d8-a7fb70a90e2d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:54.762296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:54.763455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:54.763912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:54.764358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] PATH_INFO: `/v3/users/swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:54.780148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:54.782797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:54.814465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:54.817180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:41:54.819360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:54.823102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 345/682] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:41:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:54.824825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:54.928707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:54.939329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37be9b3-1d27-4632-87d8-a7fb70a90e2d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:55.003711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dff00-8919-4585-8e30-69cf2e61bf8e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:55.006483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581868a-6b38-40eb-a4f7-b82fec76374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:55.007592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581868a-6b38-40eb-a4f7-b82fec76374d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:55.008069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581868a-6b38-40eb-a4f7-b82fec76374d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:55.008656 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581868a-6b38-40eb-a4f7-b82fec76374d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:55.009662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581868a-6b38-40eb-a4f7-b82fec76374d None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=2fc5135327fe41409cc4ba407b004d1a) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:55.010287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581868a-6b38-40eb-a4f7-b82fec76374d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:55.013396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581868a-6b38-40eb-a4f7-b82fec76374d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:55.027367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 338/683] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:54 2022] GET /identity/v3/role_assignments?user.id=2fc5135327fe41409cc4ba407b004d1a&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:55.029362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507070c-b54b-44a3-b890-c3f06eccb774 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:55.029779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507070c-b54b-44a3-b890-c3f06eccb774 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:55.030168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507070c-b54b-44a3-b890-c3f06eccb774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:55.073280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-1507070c-b54b-44a3-b890-c3f06eccb774 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:55.075710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1507070c-b54b-44a3-b890-c3f06eccb774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:55.087803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:55.089330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:55.090081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:55.090898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:55.092948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=6812dddc3db746b6893f12f1ae0c715d) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:55.093977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:55.097994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:55.111342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 346/684] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:54 2022] GET /identity/v3/role_assignments?user.id=6812dddc3db746b6893f12f1ae0c715d&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:55.114187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:55.128888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:55.234224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 339/685] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:55.273259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507070c-b54b-44a3-b890-c3f06eccb774 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:55.428882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e6c966-d19b-4f31-9577-ed0dc9a6d888 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:55.461366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:55.487905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507070c-b54b-44a3-b890-c3f06eccb774 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:55.508476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:55.509652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:55.510151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:55.510561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:55.511644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:55.512344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:55.516697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:55.557656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 347/686] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Jun 16 14:41:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:55.572084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:55.600950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:55.602020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:55.602310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:55.603181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:55.623308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:55.626059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:55.665793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:55.667518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:55.669944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:55.669944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:55.669944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:55.669944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:55.674417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 340/687] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:55.680659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:55.724761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:55.798095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb91a4-daa5-4d9c-8c96-c0c9f29a57de None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:55.852658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:55.852658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:55.853298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:55.853740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:55.933181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e3c22-355c-4cf8-af46-74be58b55840 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:56.016276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:56.017890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:56.018381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:56.018949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:56.020785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:56.021002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:56.023632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:56.031194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:56.035045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:56.086310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 341/688] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:56.100397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:56.110285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:56.161364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:56.221051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 348/689] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:41:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:41:56.224515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:56.506530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36798a-5fee-4249-9183-68f6fa04fa13 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:56.513752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a266d-5f52-4bae-9260-f513a0901e43 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:56.598366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:56.599613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:56.601383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c41f6-4176-40aa-9a29-b2e4a54e8d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:56.602827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c41f6-4176-40aa-9a29-b2e4a54e8d06 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:56.603309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c41f6-4176-40aa-9a29-b2e4a54e8d06 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:56.603791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c41f6-4176-40aa-9a29-b2e4a54e8d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:56.605163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:56.605819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:56.607254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:56.610082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:56.657467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c41f6-4176-40aa-9a29-b2e4a54e8d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:56.660747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c41f6-4176-40aa-9a29-b2e4a54e8d06 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:56.679354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 342/690] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:41:56 2022] POST /identity/v3/users => generated 254 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b53c41f6-4176-40aa-9a29-b2e4a54e8d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:56.722546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:56.724227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:56.726449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:56.726449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:56.726449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:56.726449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:56.726449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 349/691] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:56.736469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:56.969175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:57.085685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69659417-4647-4efb-b5f4-3bc0ecc72fc9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:57.101749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514953-f10d-49ea-94aa-3203354c8612 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:57.102208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514953-f10d-49ea-94aa-3203354c8612 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:57.102656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d514953-f10d-49ea-94aa-3203354c8612 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:57.105088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 350/692] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:41:57.115050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fd58f-8f14-4db3-a3c4-eb656837c931 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:57.115422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fd58f-8f14-4db3-a3c4-eb656837c931 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:57.115857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fd58f-8f14-4db3-a3c4-eb656837c931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:57.163855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7b3fd58f-8f14-4db3-a3c4-eb656837c931 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:57.166232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7b3fd58f-8f14-4db3-a3c4-eb656837c931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:41:57.169323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:57.171628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:57.172583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:57.173138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:57.175886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:57.176683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:57.178460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:57.184554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:57.198578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 343/693] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:57.207613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:57.361275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 351/694] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:41:57.466546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fd58f-8f14-4db3-a3c4-eb656837c931 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:57.478899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7213aba-f5c2-4647-965a-62d2ac1b73ab None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:57.573474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:57.602840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:57.604782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:57.606060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:57.606887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:57.624769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:57.628224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:41:57.657375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:57.659177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:41:57.661889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:41:57.666617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 344/695] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:41:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:57.677755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:57.815468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fd58f-8f14-4db3-a3c4-eb656837c931 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:57.950174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:57.951636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:57.952146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:57.952645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:57.997407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:41:58.034006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:58.036606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:58.040077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 352/696] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:41:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 572 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:41:58.046616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:58.050875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:58.073442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18251a0-34b3-47a2-b1cb-fd4339bbb491 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:58.165983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:58.167623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:58.168132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:58.168635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:58.169704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:58.170248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:58.171992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:58.208006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 345/697] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:58.218755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:58.325409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:58.326115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32259fb6-b53c-492a-9a5d-83c68d521f71 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:58.445547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:58.446944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:58.447298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:58.447787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:58.449343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:58.449883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:58.452646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:58.492522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 353/698] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:58.502870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:58.508288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae027244-5561-4c25-aeb4-bb68a3c05077 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:58.573512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:58.575964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:58.577491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:58.584173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:58.586499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=2013d4199e3841109b7c4661ff45b160) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:58.587614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:58.590436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:58.605925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 346/699] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:41:58 2022] GET /identity/v3/role_assignments?user.id=2013d4199e3841109b7c4661ff45b160&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 608 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:41:58.700581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:58.794288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ebaf9-91de-4c6e-aae7-db49e1ce4c2e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:58.853335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:58.855162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:58.855807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:58.856422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:58.872824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:58.875196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:41:58.912607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:58.914467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:58.916219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:41:58.916219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:41:58.916219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:41:58.916219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:58.921465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 354/700] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:58.925129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:59.149045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7250a-61a0-4674-b9ce-cb24bc283902 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:59.165452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:59.230129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:59.231378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:59.232150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:59.232943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:59.234774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:41:59.235687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:59.239353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:59.265199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 347/701] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:41:59.273815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:59.509599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d113d8-905c-471d-a8db-1c7b55d26ccb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:41:59.544993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:41:59.598063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721f88c-586b-44ad-be56-801c1a66d311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:41:59.599841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721f88c-586b-44ad-be56-801c1a66d311 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:59.600720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721f88c-586b-44ad-be56-801c1a66d311 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:59.601592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721f88c-586b-44ad-be56-801c1a66d311 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:59.630863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721f88c-586b-44ad-be56-801c1a66d311 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:41:59.635013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721f88c-586b-44ad-be56-801c1a66d311 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4721f88c-586b-44ad-be56-801c1a66d311 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:41:59.671045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:41:59.678207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:41:59.680575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:41:59.686251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 355/702] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:41:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:41:59.696388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:41:59.749429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875fd36-736d-44ef-be25-1199fdce94d4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:59.750874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875fd36-736d-44ef-be25-1199fdce94d4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:59.752517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875fd36-736d-44ef-be25-1199fdce94d4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:59.760990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 356/703] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:41:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:41:59.771430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb7ec3-fc5f-4ff9-9989-4e746d8423f4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:41:59.771865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb7ec3-fc5f-4ff9-9989-4e746d8423f4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:41:59.772380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb7ec3-fc5f-4ff9-9989-4e746d8423f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:41:59.835260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f9fb7ec3-fc5f-4ff9-9989-4e746d8423f4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:41:59.837737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f9fb7ec3-fc5f-4ff9-9989-4e746d8423f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:00.023396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37dbd0-b232-4dcf-8602-3f360c61176f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:00.042536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 357/704] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:42:00.045505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b174e7-315c-40f7-a28a-48ac7bf6ac82 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:00.046035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b174e7-315c-40f7-a28a-48ac7bf6ac82 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:00.046554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b174e7-315c-40f7-a28a-48ac7bf6ac82 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:00.055132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 358/705] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:00.060372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ce98-20d2-44e1-a823-0ea27cbe8cc2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:00.060945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ce98-20d2-44e1-a823-0ea27cbe8cc2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:00.061460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9ce98-20d2-44e1-a823-0ea27cbe8cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:00.110804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:00.112223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:00.113899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:00.114873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:00.117122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:00.118274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:00.121596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:00.131093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-dde9ce98-20d2-44e1-a823-0ea27cbe8cc2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:00.133398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dde9ce98-20d2-44e1-a823-0ea27cbe8cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:00.162520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 348/706] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:41:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:00.168639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:00.212917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:00.302852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 359/707] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 14:42:00.305738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9ce98-20d2-44e1-a823-0ea27cbe8cc2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:00.358453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd046e-38fd-4651-8b3a-d6e0fe5a44c8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:00.402914 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:00.419827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:00.420902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:00.421204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:00.421676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:00.422577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:00.424413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:00.455103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:00.457420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:00.459128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:00.464155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 349/708] 10.222.0.72 () {56 vars in 1129 bytes} [Thu Jun 16 14:42:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:00.466196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:00.529525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9ce98-20d2-44e1-a823-0ea27cbe8cc2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:00.602585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:00.603711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:00.604238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:00.604764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:00.662718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:00.699232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cfd09-9ecc-47aa-b67a-1ae5ce42145c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:00.737890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:00.741399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:00.766880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:00.769201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:00.770089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:00.770833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] PATH_INFO: `/v3/services/network` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:00.772479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:00.775805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:00.800908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:00.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:00.806079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:00.809015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:00.813565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 350/709] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:00 2022] GET /identity/v3/services/network => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:00.815275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:00.942567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 360/710] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 14:42:00.944579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:00.992695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc8dc2-1fd8-45e1-bdf5-f5645156cfc1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:01.056284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66564a8a-d04b-4afd-967c-7bdde2237316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:01.057421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564a8a-d04b-4afd-967c-7bdde2237316 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.057878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564a8a-d04b-4afd-967c-7bdde2237316 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.058231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564a8a-d04b-4afd-967c-7bdde2237316 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.059509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66564a8a-d04b-4afd-967c-7bdde2237316 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:01.060028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66564a8a-d04b-4afd-967c-7bdde2237316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:01.062559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66564a8a-d04b-4afd-967c-7bdde2237316 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:01.075091 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 351/711] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Jun 16 14:42:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:01.077450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd74c7-2489-440f-89d9-188993723d6e None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.078169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd74c7-2489-440f-89d9-188993723d6e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.078668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd74c7-2489-440f-89d9-188993723d6e None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.080949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 352/712] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:01.082767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd74c7-2489-440f-89d9-188993723d6e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:01.114520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:01.158517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589b9a7-9d22-468e-9d9c-4807efb9e216 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:01.232461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:01.233512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.233944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.234373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.235586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] RBAC: Adding query filter params (name=network) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:01.236080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:01.238508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:01.241959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:01.247930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 361/713] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:01.250852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:01.297067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd74c7-2489-440f-89d9-188993723d6e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:01.377549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb75b9-c31b-43b1-82f6-a6ae2e28a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:01.380454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb75b9-c31b-43b1-82f6-a6ae2e28a2da None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.380982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb75b9-c31b-43b1-82f6-a6ae2e28a2da None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.381432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb75b9-c31b-43b1-82f6-a6ae2e28a2da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.411552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb75b9-c31b-43b1-82f6-a6ae2e28a2da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:01.413895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eb75b9-c31b-43b1-82f6-a6ae2e28a2da None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-32eb75b9-c31b-43b1-82f6-a6ae2e28a2da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:01.452552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:01.453978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:01.454892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:01.454892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:01.454892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:01.454892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:01.455119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 353/714] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:01.458266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273937-469d-4fa8-b847-4fce56f53c86 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.458842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273937-469d-4fa8-b847-4fce56f53c86 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.459331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07273937-469d-4fa8-b847-4fce56f53c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.481348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef03807-631a-40ea-a1c3-3377826a45b6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:01.489514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-07273937-469d-4fa8-b847-4fce56f53c86 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:01.491724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-07273937-469d-4fa8-b847-4fce56f53c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:01.535600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:01.536751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.537287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.537670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.538462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:01.538816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:01.541106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:01.558192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 362/715] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Jun 16 14:42:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:01.565097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:01.638248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:01.657895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 354/716] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:42:01.659641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273937-469d-4fa8-b847-4fce56f53c86 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:01.725700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:01.836932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ce6aa-6997-4d03-986b-ab2ba249f995 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:01.874560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07273937-469d-4fa8-b847-4fce56f53c86 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:01.916348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:01.917556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.918049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.918529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.919958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] RBAC: Adding query filter params (type=network) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:01.920316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:01.922926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:01.932033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 363/717] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:01.937830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:01.959525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:01.961254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:01.962256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:01.962982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:01.964509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:01.965418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:01.968167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:01.991603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 355/718] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:02.025309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:02.183279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fc510-8c59-491d-8bb6-ccc4649ff1b7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:02.196600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:02.239758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:02.256524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:02.257813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:02.258277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:02.258724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:02.280153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:02.292897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fca50-b25f-41b5-997e-e17d4af42444 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:02.295621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:02.297728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:42:02.299176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:02.306549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 364/719] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:42:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:02.314866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:02.368541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:02.370567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:02.371417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:02.372131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:02.385384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:02.389266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:02.417406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:02.419032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:02.420632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:02.424009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 356/720] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:02.432130 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:02.584708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6909c-e6b2-4b82-acd0-2876bc107ccb None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:02.641673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fea7fe-7957-404e-b167-88ea45e9eadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:02.642371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fea7fe-7957-404e-b167-88ea45e9eadc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:02.642915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fea7fe-7957-404e-b167-88ea45e9eadc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:02.643188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fea7fe-7957-404e-b167-88ea45e9eadc None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:02.643745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9c670-b647-4be8-9106-63b5216796ef None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:02.644508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fea7fe-7957-404e-b167-88ea45e9eadc None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:02.644784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fea7fe-7957-404e-b167-88ea45e9eadc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:02.646564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fea7fe-7957-404e-b167-88ea45e9eadc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:02.660756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 365/721] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:42:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:02.663302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812120-ceee-4eb1-ae67-6bb8fb758294 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:02.663839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812120-ceee-4eb1-ae67-6bb8fb758294 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:02.664645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812120-ceee-4eb1-ae67-6bb8fb758294 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:02.667696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 366/722] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:02.668499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86812120-ceee-4eb1-ae67-6bb8fb758294 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:02.687511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:02.715045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821313fa-7e71-4135-af94-339d69db9cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:02.716556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821313fa-7e71-4135-af94-339d69db9cfd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:02.717336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821313fa-7e71-4135-af94-339d69db9cfd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:02.718043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821313fa-7e71-4135-af94-339d69db9cfd None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:02.719806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821313fa-7e71-4135-af94-339d69db9cfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:02.720616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821313fa-7e71-4135-af94-339d69db9cfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:02.724485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821313fa-7e71-4135-af94-339d69db9cfd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:02.749511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 357/723] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:02.752218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76d7b9-881c-4e6f-8c22-5d4067a04a9c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:02.752974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76d7b9-881c-4e6f-8c22-5d4067a04a9c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:02.753704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76d7b9-881c-4e6f-8c22-5d4067a04a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:02.768217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:02.786136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fe76d7b9-881c-4e6f-8c22-5d4067a04a9c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:02.788558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fe76d7b9-881c-4e6f-8c22-5d4067a04a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:02.872049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86812120-ceee-4eb1-ae67-6bb8fb758294 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:02.926251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:02.927236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:02.927526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:02.927806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:02.934871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 358/724] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:42:02.936908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76d7b9-881c-4e6f-8c22-5d4067a04a9c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:02.964595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:02.972604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:03.010949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:03.012626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:03.014217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:03.014217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:03.014217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:03.014217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:03.018331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 367/725] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:03.018331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:03.150133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76d7b9-881c-4e6f-8c22-5d4067a04a9c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:03.156634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:03.222867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad057641-e729-4c33-b75f-11031a52f0ab None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:03.225726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:03.229620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:03.230802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:03.231273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:03.231769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:03.233490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0b8661ae3402445c9c027e739983f26e) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:03.234086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:03.236767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:03.254815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 359/726] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:02 2022] GET /identity/v3/role_assignments?user.id=0b8661ae3402445c9c027e739983f26e&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:03.254815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:03.318756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:03.319981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:03.320476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:03.320978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:03.356976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:03.371395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:03.372422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:03.373315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:03.385558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 368/727] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:03.387474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:03.468239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff532-c483-43eb-b219-b2fe76b3f4dd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:03.540404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:03.541637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:03.542182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:03.542694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:03.544208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:03.544689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:03.547478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:03.574187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 360/728] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:03.583281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:03.644625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:03.711003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7105e9d-24b3-457a-98d2-3b845883160c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:03.834270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:03.835451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:03.835950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:03.836456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:03.837671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:03.837996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:03.839472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:03.856530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f13a02-e99c-4fe8-9bba-2acb11fd6415 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:03.862002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 369/729] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:03.870439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:03.945819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:03.949953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:03.951381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:03.951855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:03.952339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:03.967546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:03.970627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:04.002346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:04.006398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:04.012968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 361/730] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:42:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:04.015237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:04.161169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6cf0c-5cb4-4019-b976-049c1cd84b23 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:04.287736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:04.290464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:04.291269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:04.291991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:04.300378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:04.327475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:04.331178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:04.339800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119414ce-ec9f-484e-a19e-61f725b37a0b None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:04.405825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:04.407587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:04.409408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:04.409408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:04.409408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:04.409408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:04.413581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 370/731] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 544 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:04.423573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:04.484493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:04.488784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:04.492768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:04.493738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:04.495771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:04.499658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:04.506001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:04.586485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 362/732] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Jun 16 14:42:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 573 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:04.595469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:04.649598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:04.688515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19b8c1-ae1e-412e-959e-1149534c0378 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:04.755073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:04.756371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:04.756826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:04.757308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:04.758531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:04.759034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:04.761706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:04.787169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 371/733] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:04.796468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:04.993484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcee825-bb5a-4c72-8e3c-9ba146598f46 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:05.056290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:05.057701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:05.058206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:05.058698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:05.068519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:05.080232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e927d0-4a36-4379-b358-7b06b4422ec5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:05.082511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:05.084031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:05.114086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:05.116573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:05.118542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 372/734] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:05.122781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:05.162569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:05.164246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:05.165105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:05.165907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:05.200455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:05.326514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed6e2-005e-4a3f-b9b7-02877e935145 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:05.330364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:05.333834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:05.392557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:05.393988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:05.394499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:05.395503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:05.396629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:05.397212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:05.400277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:05.441299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 373/735] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Jun 16 14:42:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:05.450446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:05.590072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 363/736] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 993 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:42:05.602892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:05.616101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:05.709061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:05.751240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a4b8d-9620-425c-acae-235675a43021 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:05.839760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf542d2-d7b7-43ea-b541-693db745a221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:05.841757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf542d2-d7b7-43ea-b541-693db745a221 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:05.842493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf542d2-d7b7-43ea-b541-693db745a221 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:05.843483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf542d2-d7b7-43ea-b541-693db745a221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:05.871539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9420a20-cf21-41f4-a400-57ab4ac38eda None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:05.944760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28796ceb-96a1-4aec-8def-95d66407f31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:05.945888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28796ceb-96a1-4aec-8def-95d66407f31f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:05.946328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28796ceb-96a1-4aec-8def-95d66407f31f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:05.946785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28796ceb-96a1-4aec-8def-95d66407f31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:05.965520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28796ceb-96a1-4aec-8def-95d66407f31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:05.969028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28796ceb-96a1-4aec-8def-95d66407f31f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:05.991288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf542d2-d7b7-43ea-b541-693db745a221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:05.994542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf542d2-d7b7-43ea-b541-693db745a221 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-28796ceb-96a1-4aec-8def-95d66407f31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:06.009986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:06.011744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:06.013404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:06.013404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:06.013404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:06.013404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:06.017035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 364/737] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:06.020023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c533892-0f25-4d23-b8d1-e5b7ff00b79c None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:06.020819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c533892-0f25-4d23-b8d1-e5b7ff00b79c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:06.021489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c533892-0f25-4d23-b8d1-e5b7ff00b79c None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:06.023781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 365/738] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:06.025344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c533892-0f25-4d23-b8d1-e5b7ff00b79c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:06.113728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:06.212045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 374/739] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 758 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 14:42:06.214306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f067-1f99-41d6-827a-71e803ca0d3b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:06.215012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f067-1f99-41d6-827a-71e803ca0d3b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:06.215542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f067-1f99-41d6-827a-71e803ca0d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:06.226822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:06.238550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ef36f067-1f99-41d6-827a-71e803ca0d3b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:06.238785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c533892-0f25-4d23-b8d1-e5b7ff00b79c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:06.240240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ef36f067-1f99-41d6-827a-71e803ca0d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:06.305042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:06.306148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:06.306655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:06.307119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:06.308183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:06.308707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:06.310460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:06.326272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 366/740] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:06.327545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:06.363519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 375/741] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:42:06.366976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60b4d-1836-44d1-a38f-4f782ef588f8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:06.367773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60b4d-1836-44d1-a38f-4f782ef588f8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:06.368276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60b4d-1836-44d1-a38f-4f782ef588f8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:06.371183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 376/742] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:06.372255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60b4d-1836-44d1-a38f-4f782ef588f8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:06.527438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395b398-4dff-4af5-b8b0-8d0a284bd9d7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:06.564356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:06.574177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcdb3e-7426-48a6-86f4-5d174930902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:06.575020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcdb3e-7426-48a6-86f4-5d174930902a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:06.575310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcdb3e-7426-48a6-86f4-5d174930902a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:06.575591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcdb3e-7426-48a6-86f4-5d174930902a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:06.591548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcdb3e-7426-48a6-86f4-5d174930902a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:06.594008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcdb3e-7426-48a6-86f4-5d174930902a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:06.602946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60b4d-1836-44d1-a38f-4f782ef588f8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-58dcdb3e-7426-48a6-86f4-5d174930902a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:06.614685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:06.616423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:06.618996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:06.618996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:06.618996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:06.618996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:06.621029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 367/743] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:06.622558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189da718-b787-4183-93c9-317ae2e16f43 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:06.622855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189da718-b787-4183-93c9-317ae2e16f43 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:06.623134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189da718-b787-4183-93c9-317ae2e16f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:06.647139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-189da718-b787-4183-93c9-317ae2e16f43 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:06.649019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-189da718-b787-4183-93c9-317ae2e16f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:06.654140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:06.684401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:06.685801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:06.686251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:06.686702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:06.708798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:06.710735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:06.738585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:06.740716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:06.742723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:06.746156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 377/744] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:06.748053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:06.782355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 368/745] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 14:42:06.783832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189da718-b787-4183-93c9-317ae2e16f43 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:06.935314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0eed0-a512-4a54-9e44-b1de24b864b8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:06.990027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189da718-b787-4183-93c9-317ae2e16f43 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:06.999506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:07.000431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.000754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.001147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.001933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:07.003696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:07.038560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 378/746] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:42:06 2022] POST /identity/v3/services => generated 251 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:07.039901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:07.055899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:07.099207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:07.102112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.102570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.103026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.104453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:07.105023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:07.107673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:07.112451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:07.145565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 369/747] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:07.147145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:07.287672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3f64-34e8-4b4f-92eb-2d306a291481 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:07.343252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f866e-0391-413f-a8ab-34a1d53d177a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:07.372385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:07.373511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.373965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.374410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.375814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:07.376329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:07.380481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:07.400995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 379/748] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:07.402483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:07.421984 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e9fd9-fd98-4a12-952b-415afc9473ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:07.423099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e9fd9-fd98-4a12-952b-415afc9473ed None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.423553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e9fd9-fd98-4a12-952b-415afc9473ed None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.424829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e9fd9-fd98-4a12-952b-415afc9473ed None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.426348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e9fd9-fd98-4a12-952b-415afc9473ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:07.428939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e9fd9-fd98-4a12-952b-415afc9473ed None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:07.454728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 370/749] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:42:07 2022] POST /identity/v3/services => generated 240 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:07.463419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e61f2b-2d87-4ba2-888b-7c359c4d6bd7 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.463858 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e61f2b-2d87-4ba2-888b-7c359c4d6bd7 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.464291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e61f2b-2d87-4ba2-888b-7c359c4d6bd7 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.473034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 371/750] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:07 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:07.474491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e61f2b-2d87-4ba2-888b-7c359c4d6bd7 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:07.546671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:07.631779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a6472-ef46-4d94-a788-9e3670d326ff None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:07.635392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:07.754157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e61f2b-2d87-4ba2-888b-7c359c4d6bd7 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:07.757247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3c363-cc77-4329-849d-d7cf1ffa08e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:07.759526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3c363-cc77-4329-849d-d7cf1ffa08e6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.759526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3c363-cc77-4329-849d-d7cf1ffa08e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.759892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3c363-cc77-4329-849d-d7cf1ffa08e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.779114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3c363-cc77-4329-849d-d7cf1ffa08e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:07.782019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3c363-cc77-4329-849d-d7cf1ffa08e6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bba3c363-cc77-4329-849d-d7cf1ffa08e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:07.812639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:07.814422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:07.816810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:07.819342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 380/751] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:07.819568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa437627-1052-423a-bb8e-f5f923b11d11 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.819880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa437627-1052-423a-bb8e-f5f923b11d11 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.823967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa437627-1052-423a-bb8e-f5f923b11d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.832036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:07.832752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:07.833114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:07.833396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:07.834262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=77bd2bfa6a5343f69db3af35547a261c) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:07.834599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:07.838733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:07.845820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 372/752] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:07 2022] GET /identity/v3/role_assignments?user.id=77bd2bfa6a5343f69db3af35547a261c&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:07.846454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:07.874844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-aa437627-1052-423a-bb8e-f5f923b11d11 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:07.877800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-aa437627-1052-423a-bb8e-f5f923b11d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:08.048320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 381/753] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:08.103270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa437627-1052-423a-bb8e-f5f923b11d11 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:08.132032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1585f-738e-4531-9ce0-ea3c0b877561 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:08.170585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:08.233712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:08.234716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:08.235363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:08.235969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:08.244880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:08.244880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:08.244880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:08.265550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:08.267297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 373/754] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:08.279244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:08.375861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa437627-1052-423a-bb8e-f5f923b11d11 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:08.454468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:08.455865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:08.456413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:08.456942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:08.492546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:08.508283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:08.510249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:08.512484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 382/755] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:08.521376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:08.624273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb4bf2-6f89-44c7-a71c-dfc73560fc44 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:08.752371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdec31-bbb6-4875-9c06-c3daefacbe36 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:08.774088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:08.775145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:08.775405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:08.775599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:08.779353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:08.779353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=2013d4199e3841109b7c4661ff45b160) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:08.779353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:08.780873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:08.808588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 374/756] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:08 2022] GET /identity/v3/role_assignments?user.id=2013d4199e3841109b7c4661ff45b160&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 531 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:08.840642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:08.841744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:08.842176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:08.842609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:08.843878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:08.844362 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:08.846800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:08.876484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 383/757] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:08.888739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:08.997151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:09.134995 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9238d-338d-4418-8ef9-b72185fc0f1c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:09.203865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:09.205660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:09.206954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:09.207684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:09.230741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:09.234887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:09.272777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:09.274639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:09.276480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:09.276480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:09.276480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:09.276480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:09.280598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 375/758] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:09.291080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:09.567762 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2eac44-1a5f-4645-907f-288ce2c6136f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:09.633005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:09.634070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:09.634593 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:09.635011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:09.636264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:09.636864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:09.639407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:09.664683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 384/759] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:09.673699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:09.932235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:09.957267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39723ab-98b9-4534-b33a-f6c2f3152ae3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:10.051635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:10.053315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:10.054234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:10.055068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:10.071888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:10.075247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:10.113124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:10.114839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:42:10.116632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:10.121647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 376/760] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:42:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:10.131743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:10.214665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:10.341423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f27ea8-9613-4e9b-a731-7e860c6519e2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:10.416158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:10.417593 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:10.418109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:10.418623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:10.420350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:10.420954 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:10.423836 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:10.463326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 385/761] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:42:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:10.472516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:10.733957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2e389-cb85-47f9-a320-197836d758b8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:10.833329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:10.834771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:10.835411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:10.835929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:10.865205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:10.868144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:10.926141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:10.927119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:10.928027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:10.928027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:10.928027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:10.928027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:10.928736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 377/762] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:10.938171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:11.159082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:11.179354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80ae63-9037-4682-b355-db61d5cd7843 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:11.256026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:11.257565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:11.258283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:11.258974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:11.260785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:11.261616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:11.264545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:11.290713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 386/763] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:11.305339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:11.314904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:11.577208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce041389-5dad-4862-a581-ac1f6294b858 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:11.675896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea52734-e4d6-48eb-b021-ae19af1b7c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:11.677457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52734-e4d6-48eb-b021-ae19af1b7c1a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:11.677979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52734-e4d6-48eb-b021-ae19af1b7c1a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:11.678476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52734-e4d6-48eb-b021-ae19af1b7c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:11.695193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea52734-e4d6-48eb-b021-ae19af1b7c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:11.698135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea52734-e4d6-48eb-b021-ae19af1b7c1a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-9ea52734-e4d6-48eb-b021-ae19af1b7c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:11.735730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:11.738018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:11.740693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 378/764] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:11.750817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:11.854246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a6f96-039b-422d-8525-cd96a4701f0d None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:11.854820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a6f96-039b-422d-8525-cd96a4701f0d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:11.855135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a6f96-039b-422d-8525-cd96a4701f0d None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:11.865052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 379/765] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:11 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:11.868061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4c0b-6688-4e49-9d6b-a1589607874a None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:11.869022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4c0b-6688-4e49-9d6b-a1589607874a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:11.869809 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4c0b-6688-4e49-9d6b-a1589607874a None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:11.873584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 380/766] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:11.876389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c79459-4d2b-4dc9-8029-36c1c24ed5fd None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:11.877327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c79459-4d2b-4dc9-8029-36c1c24ed5fd None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:11.878159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c79459-4d2b-4dc9-8029-36c1c24ed5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:11.916527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-81c79459-4d2b-4dc9-8029-36c1c24ed5fd None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:11.919688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-81c79459-4d2b-4dc9-8029-36c1c24ed5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:11.990711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94264e-9815-4af2-8d94-c6f9cf123424 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:12.054765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:12.071108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:12.072754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:12.073503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:12.074191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:12.076022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:12.076784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:12.079812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:12.081249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 381/767] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:12.084405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213e43b-a8d3-4e1c-8415-7d0c88fa7591 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:12.084959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213e43b-a8d3-4e1c-8415-7d0c88fa7591 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:12.085467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213e43b-a8d3-4e1c-8415-7d0c88fa7591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:12.108060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 387/768] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:12.120750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:12.151571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-4213e43b-a8d3-4e1c-8415-7d0c88fa7591 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:12.154417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4213e43b-a8d3-4e1c-8415-7d0c88fa7591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:12.161225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:12.296995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce14b6b-d31b-4570-b8a0-b652dca4f174 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:12.297831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 382/769] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:12.298983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213e43b-a8d3-4e1c-8415-7d0c88fa7591 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:12.384215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1a591-eef0-439d-ae3d-aeb29045f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:12.385893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1a591-eef0-439d-ae3d-aeb29045f487 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:12.386366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1a591-eef0-439d-ae3d-aeb29045f487 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:12.386826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1a591-eef0-439d-ae3d-aeb29045f487 None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0b8661ae3402445c9c027e739983f26e/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:12.390410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1a591-eef0-439d-ae3d-aeb29045f487 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=0b8661ae3402445c9c027e739983f26e, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:12.393328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1a591-eef0-439d-ae3d-aeb29045f487 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:12.420892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 388/770] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:42:12 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0b8661ae3402445c9c027e739983f26e/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:42:12.420892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb3b8-8be7-419e-8dbe-96bdb8f299df None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:12.420892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb3b8-8be7-419e-8dbe-96bdb8f299df None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:12.420892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb3b8-8be7-419e-8dbe-96bdb8f299df None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:12.424063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 389/771] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:12.424063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb3b8-8be7-419e-8dbe-96bdb8f299df None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:12.521013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213e43b-a8d3-4e1c-8415-7d0c88fa7591 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:12.585679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:12.587310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:12.617572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f7f19-54c1-4c2f-83ea-1cc09fd97fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:12.619404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f7f19-54c1-4c2f-83ea-1cc09fd97fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:12.620019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f7f19-54c1-4c2f-83ea-1cc09fd97fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:12.620605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f7f19-54c1-4c2f-83ea-1cc09fd97fb3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:12.622534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f7f19-54c1-4c2f-83ea-1cc09fd97fb3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:12.625544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f7f19-54c1-4c2f-83ea-1cc09fd97fb3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-9e1f7f19-54c1-4c2f-83ea-1cc09fd97fb3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:12.652365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:12.653405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:12.654275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:12.655085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 383/772] 10.222.0.72 () {56 vars in 1129 bytes} [Thu Jun 16 14:42:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:12.657285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a633fb-6f90-41af-9e6e-fd6574c37707 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:12.657862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a633fb-6f90-41af-9e6e-fd6574c37707 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:12.658344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a633fb-6f90-41af-9e6e-fd6574c37707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:12.677826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb3b8-8be7-419e-8dbe-96bdb8f299df None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:12.689507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f2a633fb-6f90-41af-9e6e-fd6574c37707 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:12.692182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f2a633fb-6f90-41af-9e6e-fd6574c37707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:12.755876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:12.757391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:12.758014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:12.758506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] PATH_INFO: `/v3/services/network` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:12.759574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:12.762706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:12.785043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:12.786969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:12.787938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:12.791977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 390/773] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:12 2022] GET /identity/v3/services/network => generated 88 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:12.793159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:12.835212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 384/774] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:12.842681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a633fb-6f90-41af-9e6e-fd6574c37707 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:12.988647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e9d3-f629-4717-a457-6c85d04cccac None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:13.059861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:13.061408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.062005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.062634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.064013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:13.064648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:13.067863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:13.070258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:13.085388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 391/775] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Jun 16 14:42:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:13.087168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:13.097662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a633fb-6f90-41af-9e6e-fd6574c37707 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:13.139315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:13.177083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:13.179272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.180000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.180698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.183602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] RBAC: Adding query filter params (name=network) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:13.184455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:13.187678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:13.199706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 385/776] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:13.202384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:13.304058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b97a-5248-42da-8747-275350d47374 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:13.378822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:13.380106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.380607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.381104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.405292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:13.405850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bde5a-bd09-4ff7-bb02-bdb72d53c044 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:13.421500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:13.422610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:42:13.423613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:13.424722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 392/777] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:42:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:13.426532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:13.463082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafe7a5-05d7-4797-a8f6-78fbf9bc77e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:13.464119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafe7a5-05d7-4797-a8f6-78fbf9bc77e2 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.464531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafe7a5-05d7-4797-a8f6-78fbf9bc77e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.464885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafe7a5-05d7-4797-a8f6-78fbf9bc77e2 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.465789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafe7a5-05d7-4797-a8f6-78fbf9bc77e2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:13.466145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafe7a5-05d7-4797-a8f6-78fbf9bc77e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:13.467678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafe7a5-05d7-4797-a8f6-78fbf9bc77e2 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:13.476685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 386/778] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Jun 16 14:42:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:13.479101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2365-5389-4fd1-b175-c581da80bcb7 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.479603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2365-5389-4fd1-b175-c581da80bcb7 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.480221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab2365-5389-4fd1-b175-c581da80bcb7 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.488920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 387/779] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:13.488920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2365-5389-4fd1-b175-c581da80bcb7 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:13.547739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:13.577212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:13.632360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2495f79-1d7d-420d-8d02-2141c05b9591 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:13.668818 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab2365-5389-4fd1-b175-c581da80bcb7 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:13.699580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:13.700617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.701163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.701740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.702824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] RBAC: Adding query filter params (type=network) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:13.703362 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:13.705621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:13.723639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 393/780] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:13.723639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:13.745037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba83a7-0e54-4bd0-a440-98c577b16e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:13.746265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba83a7-0e54-4bd0-a440-98c577b16e84 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.746758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba83a7-0e54-4bd0-a440-98c577b16e84 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.747299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba83a7-0e54-4bd0-a440-98c577b16e84 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.748638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba83a7-0e54-4bd0-a440-98c577b16e84 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:13.749304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba83a7-0e54-4bd0-a440-98c577b16e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:13.751799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba83a7-0e54-4bd0-a440-98c577b16e84 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:13.767311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 388/781] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:13.770640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26aa03-e704-468b-a592-a977e2db0887 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:13.771148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26aa03-e704-468b-a592-a977e2db0887 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:13.771648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26aa03-e704-468b-a592-a977e2db0887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:13.802378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7a26aa03-e704-468b-a592-a977e2db0887 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:13.804538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7a26aa03-e704-468b-a592-a977e2db0887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:13.927702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56f1e-44c2-423f-a46c-b5f21e6c70fa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:13.965401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 389/782] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:13.966820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26aa03-e704-468b-a592-a977e2db0887 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:13.977289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:14.000813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1df5-5548-4862-8694-27d206264080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:14.002458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1df5-5548-4862-8694-27d206264080 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:14.004527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1df5-5548-4862-8694-27d206264080 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:14.005329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca1df5-5548-4862-8694-27d206264080 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:14.007390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca1df5-5548-4862-8694-27d206264080 None admin] RBAC: Adding query filter params (interface=public, service_id=955a0931d06c4a1c80de56ba1592b6cb, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:14.008188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca1df5-5548-4862-8694-27d206264080 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:14.012118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca1df5-5548-4862-8694-27d206264080 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:14.027683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 394/783] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:42:13 2022] GET /identity/v3/endpoints?service_id=955a0931d06c4a1c80de56ba1592b6cb&interface=public®ion_id=RegionOne => generated 194 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:14.029331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1df5-5548-4862-8694-27d206264080 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:14.044115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:14.143935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26aa03-e704-468b-a592-a977e2db0887 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:14.224125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:14.225340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:14.225926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:14.226460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:14.228069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] RBAC: Adding query filter params (interface=public, service_id=d36f7c0cc6a94e08b47154651985216d, region_id=RegionOne) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:14.228618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:14.231291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:14.256160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 390/784] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:42:13 2022] GET /identity/v3/endpoints?service_id=d36f7c0cc6a94e08b47154651985216d&interface=public®ion_id=RegionOne => generated 194 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:14.256160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:14.267121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca1df5-5548-4862-8694-27d206264080 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:14.339661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:14.341124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:14.341647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:14.342130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:14.364494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:14.367639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:14.405687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:14.407300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:14.408948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:14.408948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:14.408948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:14.408948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:14.420183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 395/785] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:14.420183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:14.470468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af5551-1024-4455-93c0-9a74a66a600d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:14.474022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:14.506960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:14.555060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:14.556467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:14.557018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:14.557529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:14.580465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:14.594671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:14.596553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 391/786] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:14.598323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:14.644245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bbb12-b258-441e-a606-7b8fab753266 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:14.718336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:14.719804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:14.720278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:14.720731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:14.722024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:14.724465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:14.740901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 396/787] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:42:14 2022] GET /identity/v3/services => generated 772 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:14.740901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:14.835089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f888e81-95db-49ae-9fd2-f7295302ce1c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:14.913964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:14.915284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:14.915853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:14.916494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:14.918009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:14.921102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:14.932919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 392/788] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:42:14 2022] GET /identity/v3/services => generated 772 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:14.938697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:14.998738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:15.025078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6daddb-0a48-4de7-8949-333d4143a390 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:15.028879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:15.101654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:15.103398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:15.104300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:15.105233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:15.107233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:15.108092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:15.111149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:15.137155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 397/789] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:15.146774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:15.178324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8485af2-721a-4113-89ea-ce530616c2bf None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:15.267323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:15.269716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:15.270162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:15.270659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:15.273309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:15.273828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:15.276477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:15.301816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 393/790] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:42:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:15.311723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:15.406836 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f52d1a-0ec7-4ed3-8554-0f7d4182708c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:15.485304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:15.486856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:15.487505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:15.488326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:15.505141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:15.508408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:15.532767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:15.547647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:15.549434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:15.551268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:15.552275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 398/791] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:42:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:15.562725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:15.566026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db427d18-a35a-4d7f-9733-4aa98063695c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:15.610064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:15.640391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2164b-6045-43a4-969f-a7dd248ad0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:15.643188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2164b-6045-43a4-969f-a7dd248ad0ae None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:15.644657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2164b-6045-43a4-969f-a7dd248ad0ae None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:15.645551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2164b-6045-43a4-969f-a7dd248ad0ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:15.666976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb2164b-6045-43a4-969f-a7dd248ad0ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:15.670267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb2164b-6045-43a4-969f-a7dd248ad0ae None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-6eb2164b-6045-43a4-969f-a7dd248ad0ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:15.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:15.708726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:15.710419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:15.710419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:15.710419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:15.710419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:15.714399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 394/792] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:15.717286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45b5a-0409-400c-aa8e-10df188a1805 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:15.718527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45b5a-0409-400c-aa8e-10df188a1805 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:15.719322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb45b5a-0409-400c-aa8e-10df188a1805 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:15.722698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 395/793] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:15.731412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a626d0-4292-4bb9-a20f-e9e7e9d275ed None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:15.731885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a626d0-4292-4bb9-a20f-e9e7e9d275ed None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:15.732405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a626d0-4292-4bb9-a20f-e9e7e9d275ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:15.761796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-56a626d0-4292-4bb9-a20f-e9e7e9d275ed None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:15.763939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-56a626d0-4292-4bb9-a20f-e9e7e9d275ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:15.771785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd0b02-30de-434e-986a-8b86a049ecbc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:15.853254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:15.855016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:15.855802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:15.856501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:15.858610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:15.859395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:15.862678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:15.907742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 399/794] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Jun 16 14:42:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:15.909709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:15.919630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 396/795] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:15.921279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a626d0-4292-4bb9-a20f-e9e7e9d275ed None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:16.066705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:16.089474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:16.108705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a626d0-4292-4bb9-a20f-e9e7e9d275ed None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:16.126060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244d1a8-e690-4d09-8400-b73b5a49e7ad None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:16.172071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:16.172813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:16.173125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:16.173462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:16.174214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:16.174526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:16.176312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:16.178847 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:16.181329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:16.182074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:16.182785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:16.203755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 400/796] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:16.203755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:16.214531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:16.218230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:16.268001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:16.270687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:16.272452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:16.272452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:16.272452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:16.272452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:16.281487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 397/797] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:16.281487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:16.428976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151fede-cdb3-4705-8747-33d660d8398b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:16.511186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8835df6-52b5-4f65-96a6-04b07d07b6d0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:16.526377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:16.528105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:16.529934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:16.530712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:16.550104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:16.558598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:16.568275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:16.580683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:16.583725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:16.586885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:16.591694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 401/798] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:16.593383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:16.611822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:16.613775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:16.615491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:16.616266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:16.635284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:16.640110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:16.680983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:16.683559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 398/799] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:16.693615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:16.818716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dd640-f341-4611-9234-4e9c56313dd6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:16.898959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:16.899346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a095b6-c0a7-4460-ad92-5f01cbe64922 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:16.900696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:16.901532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:16.902202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:16.903992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:16.905591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:16.908438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:16.935819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 402/800] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:16.937795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:16.963733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:16.965161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:16.965908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:16.966535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:16.967606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:16.968271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:16.971202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:16.992422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 399/801] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:42:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:16.994180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:17.073071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:17.084967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:17.169151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6c395-33cd-4335-bc89-be858e2f5d9a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:17.205611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21d19a-4eb0-4d64-869a-52aa7e8a0db1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:17.246208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:17.247813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:17.248626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:17.249708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:17.251529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:17.252435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:17.255961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:17.280030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:17.281471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:17.281973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:17.282476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:17.290025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 403/802] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Jun 16 14:42:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:17.291615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:17.297346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:17.300381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:17.329682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:17.330705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:17.332210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:17.333290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 400/803] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:17.334633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:17.439100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ad91-1623-435a-85b8-eb393b1630b3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:17.493719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:17.517654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:17.519694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:17.520170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:17.521228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:17.537795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abba4b6-0914-4b07-bc78-6e32c38277c5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:17.542182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:17.542378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:17.545307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:17.583116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:17.584109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:17.585044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:17.585044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:17.585044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:17.585044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:17.593698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 404/804] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:17.595283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:17.597981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:17.599281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:17.599874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:17.600659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:17.617383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:17.620345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:17.657560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:17.658601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:17.659530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:17.659530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:17.659530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:17.659530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:17.660267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 401/805] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:17.661997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:17.803898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f950a7-a98c-4fb7-8c4b-a332d714bd99 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:17.871887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7a7fe-648d-4451-9334-b6738c0139e0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:17.896564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:17.898006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:17.899334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:17.899334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:17.901381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:17.902029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:17.904955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:17.934364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 405/806] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:17.936218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:17.958485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:17.961151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:17.961890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:17.962619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:17.964929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:17.966156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:17.969531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:18.007774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 402/807] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:18.007774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:18.010527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:18.054456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:18.159782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ddcb6-9923-4b4b-b147-57a42216c7d0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:18.240204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:18.241455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.242023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.242522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.243579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:18.244236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:18.246031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:18.249903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e099c48-95fa-4353-b0b6-a8702566295e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:18.270806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 406/808] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:18.273131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:18.327621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:18.328967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.329580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.329833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/77bd2bfa6a5343f69db3af35547a261c/roles/5a44c123efd54f14bc8e76881f748f8b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.356160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=77bd2bfa6a5343f69db3af35547a261c, role_id=5a44c123efd54f14bc8e76881f748f8b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:18.358244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:18.380007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 403/809] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:42:18 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/77bd2bfa6a5343f69db3af35547a261c/roles/5a44c123efd54f14bc8e76881f748f8b => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:42:18.381429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:18.449169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:18.478164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f22ce8-4edc-4440-91df-11f28faf4551 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:18.511489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:18.558575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d77bd-b331-4a19-bb3e-c293f0c6e21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:18.560073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d77bd-b331-4a19-bb3e-c293f0c6e21e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.560602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d77bd-b331-4a19-bb3e-c293f0c6e21e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.561228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d77bd-b331-4a19-bb3e-c293f0c6e21e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.577533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d77bd-b331-4a19-bb3e-c293f0c6e21e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:18.580576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d77bd-b331-4a19-bb3e-c293f0c6e21e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:18.581426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ba42-9a9d-4a66-93f7-94e23ac46048 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b26d77bd-b331-4a19-bb3e-c293f0c6e21e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:18.610653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:18.613221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 407/810] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:42:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3def98-48cb-4776-b232-fe1ab7b99d66 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.618814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3def98-48cb-4776-b232-fe1ab7b99d66 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.623337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3def98-48cb-4776-b232-fe1ab7b99d66 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.623337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 408/811] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:18.625130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a22cd-f055-47e7-929d-1f3c32c3d0c8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.625130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a22cd-f055-47e7-929d-1f3c32c3d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.625130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a22cd-f055-47e7-929d-1f3c32c3d0c8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.628886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 409/812] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:18.629992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a22cd-f055-47e7-929d-1f3c32c3d0c8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:18.667654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d144d8a-ef3d-4429-8156-3f713af7b526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:18.669234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d144d8a-ef3d-4429-8156-3f713af7b526 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.669849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d144d8a-ef3d-4429-8156-3f713af7b526 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.670347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d144d8a-ef3d-4429-8156-3f713af7b526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.685056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d144d8a-ef3d-4429-8156-3f713af7b526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:18.687957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d144d8a-ef3d-4429-8156-3f713af7b526 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-6d144d8a-ef3d-4429-8156-3f713af7b526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:18.707713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:18.709453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 404/813] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55edcc2-7c24-41cc-8797-071b7204d17b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.711829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55edcc2-7c24-41cc-8797-071b7204d17b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.714290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55edcc2-7c24-41cc-8797-071b7204d17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.737946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-d55edcc2-7c24-41cc-8797-071b7204d17b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:18.740222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d55edcc2-7c24-41cc-8797-071b7204d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:18.792536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a22cd-f055-47e7-929d-1f3c32c3d0c8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:18.860292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:18.861335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.862504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.862953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.864884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:18.865396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:18.870177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:18.898605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 405/814] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:18.900669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c131da-6479-4708-9efc-f1df9dd86a26 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:18.902327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c131da-6479-4708-9efc-f1df9dd86a26 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:18.902813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c131da-6479-4708-9efc-f1df9dd86a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:18.912999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 410/815] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:42:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:18.912999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:18.931245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:18.938905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-38c131da-6479-4708-9efc-f1df9dd86a26 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:18.941653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38c131da-6479-4708-9efc-f1df9dd86a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:19.042439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:19.084896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 406/816] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:19.084896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c131da-6479-4708-9efc-f1df9dd86a26 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:19.242237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c131da-6479-4708-9efc-f1df9dd86a26 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:19.246242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821b2c8-f4fd-465b-9a19-9b1f5d6202d2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:19.306266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:19.307509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:19.307985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:19.308457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:19.324447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:19.325471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:19.325905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:19.326350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:19.327945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:19.328450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:19.330661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:19.356904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 411/817] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:19.356904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:19.417490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:19.442681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:19.445336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:19.481818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:19.539021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c59bf8-1a59-4780-91fc-c29e6b74eae9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:19.605533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:19.606998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:19.607787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:19.608313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:19.609637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:19.612311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:19.625639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 407/818] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:42:19.625639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:19.644798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:19.645896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 14:42:19.646888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:19.648312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 412/819] 10.222.0.72 () {56 vars in 1129 bytes} [Thu Jun 16 14:42:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:19.649952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:19.797870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:19.843216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879c0e-9be4-46c9-a9bb-a06eb73bfb4c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:19.885551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798475c5-c1eb-482b-80cb-bd7d2be1c431 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:19.923606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:19.925166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:19.925693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:19.926210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] PATH_INFO: `/v3/services/network` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:19.927619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:19.930410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:19.953449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:19.954332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:19.957036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:19.958992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 408/820] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:19 2022] GET /identity/v3/services/network => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:19.963519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:19.973547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:19.974774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:19.975019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:19.975425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/2013d4199e3841109b7c4661ff45b160/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:19.979017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=2013d4199e3841109b7c4661ff45b160, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:19.981164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:20.005505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 413/821] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:42:19 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/2013d4199e3841109b7c4661ff45b160/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:42:20.006887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:20.197908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5a0a1-62e0-4a2a-a67b-83ac927a4c72 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:20.289918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adca60-f6e1-4d80-be28-2674113674c0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:20.290261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:20.291424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:20.291782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:20.292055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:20.308624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:20.310303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:20.343948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:20.346071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:20.348229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:20.348229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:20.348229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:20.348229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:20.348229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 409/822] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:20.352886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:20.356572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:20.366958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:20.368225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:20.368983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:20.369533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:20.371471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:20.372236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:20.375304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:20.385896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 414/823] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Jun 16 14:42:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:20.396247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:20.511484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:20.546121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39e48c-862e-408e-be3c-fc37ecbe0cb1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:20.589292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa233a1-70ae-4fec-ad77-5542ecbe1f54 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:20.624469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:20.625303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:20.625614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:20.626456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:20.627247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] RBAC: Adding query filter params (name=network) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:20.627768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:20.630118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:20.640888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 410/824] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:20.640888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:20.656408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:20.658055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:20.659007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:20.659941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:20.661755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:20.662591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:20.665956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:20.701491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 415/825] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:20.704138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:20.764545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:20.827899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843afc4-dcad-44fb-bbf5-270e9f6c1a45 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:20.879585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:20.880807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:20.881208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:20.881740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:20.882581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:20.883246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:20.885843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:20.898991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 411/826] 10.222.0.72 () {56 vars in 1124 bytes} [Thu Jun 16 14:42:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:20.898991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:20.967803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d2106-2e92-4910-8de2-5abc239cdf1c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:21.007963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:21.053757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:21.055981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:21.057077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:21.057960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:21.059685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] RBAC: Adding query filter params (type=network) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:21.060788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:21.064112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:21.085024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 416/827] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:21.087386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:21.100612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c4e65-f020-4321-85f4-3d71a89c799e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:21.143201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:21.151976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:21.152864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:21.153173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:21.153468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:21.162535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:21.164201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:21.185935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:21.187107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:21.188296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:21.188769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 412/828] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:21.189957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:21.285171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30319708-58a6-4d0c-9c0b-63a184d00384 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:21.354352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:21.356143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:21.356758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:21.357451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:21.358493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:21.360473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:21.371750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88089ea-956b-4a4c-89c1-674603d5efd5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:21.401688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 417/829] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:42:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:21.404138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:21.433100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:21.433815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:21.434108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:21.434393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:21.435202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:21.441168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:21.484275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 413/830] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:42:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:21.530540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:21.656191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0b6e0-7a1d-4be4-b81d-5f7740417632 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:21.748532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a830b5a-cd82-43bb-91c2-7f38aa6ff60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:21.749848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a830b5a-cd82-43bb-91c2-7f38aa6ff60f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:21.750399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a830b5a-cd82-43bb-91c2-7f38aa6ff60f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:21.750930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a830b5a-cd82-43bb-91c2-7f38aa6ff60f None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:21.752655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a830b5a-cd82-43bb-91c2-7f38aa6ff60f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:21.753838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a830b5a-cd82-43bb-91c2-7f38aa6ff60f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:21.756987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a830b5a-cd82-43bb-91c2-7f38aa6ff60f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:21.786134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 418/831] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:21.796786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:21.909510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:22.020312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf3bc8-42a5-492c-992f-c62d678366bf None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:22.088597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff53e0-d4d9-4364-8f4a-dbf1720aeba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:22.090170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff53e0-d4d9-4364-8f4a-dbf1720aeba0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:22.090896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff53e0-d4d9-4364-8f4a-dbf1720aeba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:22.091588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff53e0-d4d9-4364-8f4a-dbf1720aeba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:22.093409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff53e0-d4d9-4364-8f4a-dbf1720aeba0 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0b8661ae3402445c9c027e739983f26e) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:22.094179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff53e0-d4d9-4364-8f4a-dbf1720aeba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:22.097374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff53e0-d4d9-4364-8f4a-dbf1720aeba0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:22.109992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 414/832] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:21 2022] GET /identity/v3/role_assignments?user.id=0b8661ae3402445c9c027e739983f26e&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:22.224802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4769a-539c-4da2-b0e4-2b89204d2b28 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:22.225823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4769a-539c-4da2-b0e4-2b89204d2b28 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:22.226547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4769a-539c-4da2-b0e4-2b89204d2b28 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:22.229677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 419/833] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:22.238265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f8a34-4611-49e7-af72-e3d984438828 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:22.239245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f8a34-4611-49e7-af72-e3d984438828 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:22.240065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f8a34-4611-49e7-af72-e3d984438828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:22.267202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5b2f8a34-4611-49e7-af72-e3d984438828 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:22.270470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5b2f8a34-4611-49e7-af72-e3d984438828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:22.364752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 415/834] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:22.390681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4769a-539c-4da2-b0e4-2b89204d2b28 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:22.562284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:22.579380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4769a-539c-4da2-b0e4-2b89204d2b28 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:22.648003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:22.649467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:22.649955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:22.650405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:22.672339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:22.687067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:22.689073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:22.691402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 420/835] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:22.703613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f8a34-4611-49e7-af72-e3d984438828 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:22.791960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:22.931094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f8a34-4611-49e7-af72-e3d984438828 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:23.009448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c025-d518-44b2-8afc-1128ac911f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:23.010598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c025-d518-44b2-8afc-1128ac911f59 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:23.011091 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c025-d518-44b2-8afc-1128ac911f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:23.011806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c025-d518-44b2-8afc-1128ac911f59 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:23.013080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37c025-d518-44b2-8afc-1128ac911f59 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:23.013629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37c025-d518-44b2-8afc-1128ac911f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:23.016309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37c025-d518-44b2-8afc-1128ac911f59 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:23.034088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 416/836] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:42:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:23.040388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:23.233834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69e9dc-cd9d-445e-b588-d9a72f0f1b1c None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:23.234813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69e9dc-cd9d-445e-b588-d9a72f0f1b1c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:23.235058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69e9dc-cd9d-445e-b588-d9a72f0f1b1c None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:23.239701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 417/837] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:23.246792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f5ee-dfc7-4603-aedd-1ec6b6a0e5d9 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:23.247064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f5ee-dfc7-4603-aedd-1ec6b6a0e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:23.247354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49f5ee-dfc7-4603-aedd-1ec6b6a0e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:23.256282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b44893-6caf-4824-808a-390b2732f053 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:23.279153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7f49f5ee-dfc7-4603-aedd-1ec6b6a0e5d9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:23.281922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7f49f5ee-dfc7-4603-aedd-1ec6b6a0e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:23.321408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:23.322927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:23.323462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:23.323936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:23.346320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:23.350004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:23.389424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:23.391112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:23.393156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:23.393156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:23.393156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:23.393156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:23.393156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 421/838] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:23.402440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:23.408957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:23.438066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 418/839] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:23.488344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49f5ee-dfc7-4603-aedd-1ec6b6a0e5d9 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:23.533852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:23.578485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6df9ac-dcfd-4237-98d0-629b8009ff75 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:23.656886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:23.658569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:23.659509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:23.660404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:23.662197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:23.663216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:23.666524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:23.670952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49f5ee-dfc7-4603-aedd-1ec6b6a0e5d9 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:23.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 422/840] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:23.703592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:23.731173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:23.732171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:23.732628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:23.733082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:23.753592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:23.765421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:23.766913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:42:23.768194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:23.773738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 419/841] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:23.774923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:23.828457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:23.916372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3d4ec-4601-485a-ac42-6a40aac840a1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:23.961535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:23.985097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:23.986625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:23.987097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:23.988140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:23.990763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51898448-0867-4089-8cab-03a2c5cc622b None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:24.004360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:24.006553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:24.042336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:24.044531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:24.045829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 423/842] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:42:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:24.054403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:24.073674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:24.075424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:24.076238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:24.077119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:24.078760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:24.079596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:24.083099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:24.098949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 420/843] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:24.106470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:24.252495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f160-13f5-4792-9abe-74612af88665 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:24.319598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:24.346407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe6388-4efc-420e-b2cf-4cddbe2ef1e8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:24.348013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:24.350096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:24.351009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:24.353322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:24.355300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:24.356157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:24.359217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:24.406458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 424/844] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Jun 16 14:42:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:24.416908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:24.431353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:24.433208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:24.434073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:24.434970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:24.459001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:24.462397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:24.505051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:24.506904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:24.508826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:24.508826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:24.508826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:24.508826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:24.521132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 421/845] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:24.525814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:24.533364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:24.637370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bee4f-6530-4b4d-a332-a43800e8d715 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:24.695758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:24.696904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:24.697452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:24.698050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:24.766627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:24.776927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646b434-e632-45b6-aa19-1281b83d8852 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:24.803279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:24.807369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:24.835185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:24.836266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:24.836816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:24.837342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:24.838402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:24.839054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:24.841116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:24.863117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 422/846] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:24.866104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:24.996316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 425/847] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:42:24.999874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:25.006286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:25.084038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657c0c9-1176-41f2-89f7-0c6cd65c2f7e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:25.169549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:25.171007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:25.171478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:25.172367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:25.191856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:25.195189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:25.199510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:25.236064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc115573-e1ff-480a-a297-c663e012a055 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:25.238989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:25.241180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 14:42:25.243415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:25.258940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 423/848] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:42:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:25.270156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:25.326310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8adcd4-71bd-40d6-ab62-b144b4ed719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:25.328569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8adcd4-71bd-40d6-ab62-b144b4ed719c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:25.330156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8adcd4-71bd-40d6-ab62-b144b4ed719c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:25.331416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8adcd4-71bd-40d6-ab62-b144b4ed719c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:25.357229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8adcd4-71bd-40d6-ab62-b144b4ed719c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:25.361193 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8adcd4-71bd-40d6-ab62-b144b4ed719c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8f8adcd4-71bd-40d6-ab62-b144b4ed719c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:25.406942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:25.409831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:25.411489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:25.411489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:25.411489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:25.411489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:25.411922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 426/849] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:25.414093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a5e5-d673-462e-9d20-be5d4ee70bf7 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:25.414488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a5e5-d673-462e-9d20-be5d4ee70bf7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:25.414894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9a5e5-d673-462e-9d20-be5d4ee70bf7 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:25.417705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 427/850] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:25.423083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9a5e5-d673-462e-9d20-be5d4ee70bf7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:25.505117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711fed0-e5f6-425c-9a79-0503927a2584 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:25.553532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:25.580761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4e3ec-c33c-4cc3-9a52-3a84dbe13477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:25.582541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4e3ec-c33c-4cc3-9a52-3a84dbe13477 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:25.583055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4e3ec-c33c-4cc3-9a52-3a84dbe13477 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:25.583551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4e3ec-c33c-4cc3-9a52-3a84dbe13477 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:25.585237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4e3ec-c33c-4cc3-9a52-3a84dbe13477 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:25.585817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4e3ec-c33c-4cc3-9a52-3a84dbe13477 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:25.588633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4e3ec-c33c-4cc3-9a52-3a84dbe13477 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:25.616969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 424/851] 10.222.0.72 () {56 vars in 1135 bytes} [Thu Jun 16 14:42:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:25.618552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb7a0-67f6-4be1-9a23-0fb4f2933287 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:25.618958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb7a0-67f6-4be1-9a23-0fb4f2933287 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:25.619312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb7a0-67f6-4be1-9a23-0fb4f2933287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:25.639155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-597cb7a0-67f6-4be1-9a23-0fb4f2933287 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:25.640400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-597cb7a0-67f6-4be1-9a23-0fb4f2933287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:25.665106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9a5e5-d673-462e-9d20-be5d4ee70bf7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:25.737540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:25.739817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:25.740318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:25.740799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:25.742290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:25.750251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 425/852] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:25.750561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:25.750788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb7a0-67f6-4be1-9a23-0fb4f2933287 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:25.750946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:25.750946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:25.777470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 428/853] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:25.787262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:25.911619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb7a0-67f6-4be1-9a23-0fb4f2933287 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:25.927676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:25.960461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252f38d-2697-4a5f-90a2-4c09788de848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:25.961308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f38d-2697-4a5f-90a2-4c09788de848 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:25.961699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f38d-2697-4a5f-90a2-4c09788de848 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:25.962070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252f38d-2697-4a5f-90a2-4c09788de848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.001224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a803142-dfc6-4506-a913-dde470508ad2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:26.053024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252f38d-2697-4a5f-90a2-4c09788de848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:26.054796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252f38d-2697-4a5f-90a2-4c09788de848 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:26.069084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:26.070126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:26.070588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:26.070959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.071828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:26.073493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:26.087209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:26.089462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:26.091191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 429/854] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:26.092754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:26.186237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:26.229638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 426/855] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:42:26.231298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6089419-dae7-4222-9f1c-38c64a0e9bcd None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:26.231588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6089419-dae7-4222-9f1c-38c64a0e9bcd None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:26.231895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6089419-dae7-4222-9f1c-38c64a0e9bcd None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.236884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 427/856] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:26.236884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6089419-dae7-4222-9f1c-38c64a0e9bcd None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:26.249507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af4494-8d9b-48e4-8b89-f6e85d4fece6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:26.258065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:26.303718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:26.305259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:26.305735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:26.306199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.328119 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:26.329699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:26.347026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:26.348814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 430/857] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:26.350622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:26.484642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b940c09-518d-420c-b7c4-4c3a0dfa8ce9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:26.509587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:26.545610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd500118-8ef0-4e0f-9672-432a7d0fb24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:26.547084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd500118-8ef0-4e0f-9672-432a7d0fb24c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:26.547548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd500118-8ef0-4e0f-9672-432a7d0fb24c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:26.548030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd500118-8ef0-4e0f-9672-432a7d0fb24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.565062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd500118-8ef0-4e0f-9672-432a7d0fb24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:26.568011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd500118-8ef0-4e0f-9672-432a7d0fb24c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bd500118-8ef0-4e0f-9672-432a7d0fb24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:26.604196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:26.606242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:26.608043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:26.608043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:26.608043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:26.608043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:26.608478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6089419-dae7-4222-9f1c-38c64a0e9bcd None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:26.609208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 431/858] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:26.609208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d6f84-8614-427e-977e-bf7f8e83baff None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:26.609985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d6f84-8614-427e-977e-bf7f8e83baff None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:26.610247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d6f84-8614-427e-977e-bf7f8e83baff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.639777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a76d6f84-8614-427e-977e-bf7f8e83baff None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:26.641852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a76d6f84-8614-427e-977e-bf7f8e83baff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:26.673302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:26.674407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:26.674931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:26.675417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.676877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:26.677436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:26.680187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:26.689959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 428/859] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:26.691567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:26.766895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 432/860] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:26.769229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d6f84-8614-427e-977e-bf7f8e83baff None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:26.843864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:26.894675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b125bb-c5e5-4025-8b76-72688c410ebf None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:26.958564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:26.959599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:26.960065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:26.960538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:26.961925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:26.962404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:26.964757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:26.967162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:26.972483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d6f84-8614-427e-977e-bf7f8e83baff None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:26.987626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 429/861] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:26.989356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:27.037576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:27.038522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:27.039028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:27.039528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:27.041027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:27.041565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:27.044224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:27.067649 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 433/862] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:27.068960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:27.197075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d2f0c-617a-41ee-a35f-313a46aa15bd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:27.262611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:27.263859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:27.264378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:27.264914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:27.266345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:27.266919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:27.270519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:27.273222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:27.282066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 430/863] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:27.282351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db55452-f711-491d-bec7-e8cc1cf3a10a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:27.284078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:27.360956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:27.361863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:27.362720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:27.363061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:27.363826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:27.365491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:27.380196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:27.382296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:27.384264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 434/864] 10.222.0.72 () {56 vars in 1132 bytes} [Thu Jun 16 14:42:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:27.387557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:27.427479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:27.501959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd049e-e21d-4d19-ba96-db92d84c77e3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:27.572452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9b6ef-88d8-48b7-b2e4-2c0e5c8a121b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:27.577212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:27.578518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:27.579038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:27.579553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:27.581827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=77bd2bfa6a5343f69db3af35547a261c) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:27.582411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:27.585211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:27.597742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 431/865] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:27 2022] GET /identity/v3/role_assignments?user.id=77bd2bfa6a5343f69db3af35547a261c&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:27.599349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:27.653135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:27.655244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:27.655932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:27.656585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:27.672521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:27.676042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:27.708569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:27.710655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:27.712808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:27.715725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 435/866] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:27.725949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:27.738909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:27.829178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f50414-4838-47d7-90ca-9109821e5263 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:27.877564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:27.920635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fcdcd-b1cd-431e-bdd7-b57e445296b6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:27.926736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:27.928275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:27.929058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:27.929791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:27.931280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:27.932131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:27.934610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:27.945260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 432/867] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:27.954069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:27.996273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:27.997156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:27.997441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:27.997714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:27.998644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:27.999182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:28.000812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:28.016616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 436/868] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:28.026057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:28.173767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b300f1-3882-4d29-b1bc-159ca0dd9d06 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:28.230354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b10e6-e5e9-4e70-be9f-b0b0e4d71c6c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:28.232947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:28.254056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6901150-e4e8-4fb5-8ae3-aa2029f1bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:28.255226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6901150-e4e8-4fb5-8ae3-aa2029f1bde5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:28.255732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6901150-e4e8-4fb5-8ae3-aa2029f1bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:28.256780 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6901150-e4e8-4fb5-8ae3-aa2029f1bde5 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:28.258590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6901150-e4e8-4fb5-8ae3-aa2029f1bde5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:28.259141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6901150-e4e8-4fb5-8ae3-aa2029f1bde5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:28.262078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6901150-e4e8-4fb5-8ae3-aa2029f1bde5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:28.279176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 433/869] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:28.317549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23883907-451d-4559-8d3a-9d7801db21e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:28.318860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23883907-451d-4559-8d3a-9d7801db21e9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:28.319330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23883907-451d-4559-8d3a-9d7801db21e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:28.319795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23883907-451d-4559-8d3a-9d7801db21e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:28.321586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23883907-451d-4559-8d3a-9d7801db21e9 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=2013d4199e3841109b7c4661ff45b160) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:28.322119 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23883907-451d-4559-8d3a-9d7801db21e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:28.325013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23883907-451d-4559-8d3a-9d7801db21e9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:28.339144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 437/870] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:28 2022] GET /identity/v3/role_assignments?user.id=2013d4199e3841109b7c4661ff45b160&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:30.961340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfde9b-29a8-4b27-aeb8-52897e4cb38d None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:30.962294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfde9b-29a8-4b27-aeb8-52897e4cb38d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:30.962294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfde9b-29a8-4b27-aeb8-52897e4cb38d None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:30.966650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 434/871] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:30.975819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873e67e-8fb2-4cd8-97ca-82dfc1411c80 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:30.976609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873e67e-8fb2-4cd8-97ca-82dfc1411c80 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:30.977011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873e67e-8fb2-4cd8-97ca-82dfc1411c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:31.007426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e873e67e-8fb2-4cd8-97ca-82dfc1411c80 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:31.009423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e873e67e-8fb2-4cd8-97ca-82dfc1411c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:31.013443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:31.158031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 438/872] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:31.196016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfde9b-29a8-4b27-aeb8-52897e4cb38d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:31.400515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220c71-3122-4865-ae33-db66cc32b307 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:31.401057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220c71-3122-4865-ae33-db66cc32b307 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:31.401536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220c71-3122-4865-ae33-db66cc32b307 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:31.404202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 439/873] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:31.412905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf9543-a943-4c49-abd5-ab2e17eba630 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:31.413307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf9543-a943-4c49-abd5-ab2e17eba630 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:31.413763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf9543-a943-4c49-abd5-ab2e17eba630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:31.418375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfde9b-29a8-4b27-aeb8-52897e4cb38d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:31.443697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7ebf9543-a943-4c49-abd5-ab2e17eba630 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:31.446147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7ebf9543-a943-4c49-abd5-ab2e17eba630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:31.493500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:31.494807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:31.495313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:31.495801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:31.497150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:31.498790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:31.517099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 435/874] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:42:31 2022] POST /identity/v3/services => generated 251 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:31.619511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 440/875] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:31.656347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:31.772430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:31.854563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7ed73-4c74-4f89-a853-179e43acf079 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:31.921753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:31.923135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:31.923617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:31.924086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:31.945021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:31.958728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89438f-9e4b-4742-bd6b-a733afc83050 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:31.959136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89438f-9e4b-4742-bd6b-a733afc83050 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:31.959576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89438f-9e4b-4742-bd6b-a733afc83050 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:31.961569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:31.963022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:42:31.963943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:31.965564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 441/876] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:31.965641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 436/877] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Jun 16 14:42:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:31.974785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d827ef-2b42-46b2-903d-53af5683fe91 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:31.975538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:31.976106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d827ef-2b42-46b2-903d-53af5683fe91 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:31.977465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d827ef-2b42-46b2-903d-53af5683fe91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:31.997933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-00d827ef-2b42-46b2-903d-53af5683fe91 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:32.003143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-00d827ef-2b42-46b2-903d-53af5683fe91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:32.146485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be154aa-d640-4916-bcc6-6c98d2567946 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:32.164691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:32.169753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:32.183478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 442/878] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:32.186087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce667-155a-4331-9da4-535b90f067f9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:32.186603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce667-155a-4331-9da4-535b90f067f9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:32.187094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce667-155a-4331-9da4-535b90f067f9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:32.200338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 443/879] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:32 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:32.209162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b88c9c-a96f-432a-afbd-601e700acb60 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:32.209933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b88c9c-a96f-432a-afbd-601e700acb60 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:32.210589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b88c9c-a96f-432a-afbd-601e700acb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:32.228946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:32.230257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:32.230735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:32.231197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:32.233444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:32.233965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:32.236521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:32.241222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a0b88c9c-a96f-432a-afbd-601e700acb60 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:32.243325 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a0b88c9c-a96f-432a-afbd-601e700acb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:32.253273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 437/880] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Jun 16 14:42:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:32.255458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:32.393754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 444/881] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:32.396453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b88c9c-a96f-432a-afbd-601e700acb60 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:32.488784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1977d-513f-4b4a-9765-569dd1f939e0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:32.577276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:32.578489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:32.578992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:32.579469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:32.582759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:32.587164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:32.619585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 438/882] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:42:32 2022] POST /identity/v3/services => generated 269 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:32.619585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:32.650676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b88c9c-a96f-432a-afbd-601e700acb60 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:32.682278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:32.706654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:32.741374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:32.742819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:32.743696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:32.744463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:32.767359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:32.771735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:32.809442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:32.811219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:32.813017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:32.813017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:32.813017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:32.813017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:32.825221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 445/883] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:32.831459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:32.840393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c06377-4872-46bb-b4cc-0b4c191ed65d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:32.907689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:32.909668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:32.910213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:32.910980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:32.911802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:32.913720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:32.935241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:32.937315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:32.939033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 439/884] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:32.945462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:33.067524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cdb327-0860-482d-a64c-7ae8ca96448b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:33.119729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:33.136949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4337157-9f6a-4c0a-a6bc-e4acc9efd7af None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:33.145009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:33.146094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:33.146552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:33.147001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:33.148295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:33.148811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:33.151428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:33.184925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 446/885] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:33.189850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:33.218448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:33.219528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:33.220022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:33.220517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:33.222006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:33.222533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:33.225243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:33.235596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 440/886] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:33.246752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:33.270581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:33.419742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40478358-8847-462d-9d63-fb5d7b6333e6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:33.452633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d99594-3d54-4ed4-8df1-7019c3330851 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:33.494338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:33.495688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:33.496181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:33.496687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:33.513055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:33.516249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:33.526681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:33.527929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:33.528448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:33.528991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:33.530408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:33.530972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:33.533775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:33.543688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 441/887] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:33.556817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:33.559350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:33.562786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 447/888] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:42:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:33.565633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:33.617554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:33.744697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed8e55-de84-49b8-a03c-05addd62dd66 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:33.801483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:33.802549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:33.803004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:33.803459 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:33.805087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:33.805485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:33.807156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:33.834857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 442/889] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Jun 16 14:42:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:33.844407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:34.053834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:34.060169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7846f-1bc5-4657-9e5b-3558be5439d7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:34.101628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:34.102638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:34.102638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:34.102903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:34.197702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:34.199235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:34.310270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:34.324211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 448/890] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:33 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:42:34.332758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:34.453575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:34.473706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd6ee-a63b-4326-8a36-bf425f6c85d9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:34.518831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91663e-e26f-4d6b-99cb-23babf780ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:34.520192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91663e-e26f-4d6b-99cb-23babf780ca5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:34.520597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91663e-e26f-4d6b-99cb-23babf780ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:34.520917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91663e-e26f-4d6b-99cb-23babf780ca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:34.535789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91663e-e26f-4d6b-99cb-23babf780ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:34.537358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91663e-e26f-4d6b-99cb-23babf780ca5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-6f91663e-e26f-4d6b-99cb-23babf780ca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:34.558643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:34.560199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:34.561898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:34.561898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:34.561898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:34.561898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:34.561898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 443/891] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:34.567779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:34.762487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d04a87-4648-42f9-986f-2622b949c6cf None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:34.762750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d04a87-4648-42f9-986f-2622b949c6cf None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:34.763044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d04a87-4648-42f9-986f-2622b949c6cf None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:34.764698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 444/892] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:34.770787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7c27-14e9-41e5-9e56-bf3a3cc87a2b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:34.771122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7c27-14e9-41e5-9e56-bf3a3cc87a2b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:34.771340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7c27-14e9-41e5-9e56-bf3a3cc87a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:34.774545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a183e-72af-4830-b568-3d024bb91604 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:34.789976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-6c1f7c27-14e9-41e5-9e56-bf3a3cc87a2b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:34.791305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6c1f7c27-14e9-41e5-9e56-bf3a3cc87a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:34.837574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:34.838556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:34.839039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:34.839516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:34.840870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:34.841447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:34.844008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:34.859624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 449/893] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:34.870112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:34.910513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 445/894] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:34.939827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7c27-14e9-41e5-9e56-bf3a3cc87a2b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:34.976884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:35.045083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:35.063263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151ee53-6ccf-48b6-b05e-32924bc93fbf None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:35.104352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7c27-14e9-41e5-9e56-bf3a3cc87a2b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:35.118080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:35.120269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:35.120806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:35.121385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:35.134358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:35.137792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:35.158496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:35.160191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:35.160527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:35.161023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:35.161716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:35.162126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:35.163395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:35.164416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:35.164868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:35.166867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 450/895] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:35.177715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:35.185684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:35.187872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:35.190061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 446/896] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:35.199263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:35.348629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a3c29-2a5c-4a84-a99b-195e27f537ca None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:35.362947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4b2e2-4352-48bd-bc23-fd49704653b4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:35.375835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:35.407700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:35.410237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:35.411449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:35.411924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:35.412433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:35.414016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:35.414813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:35.416509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:35.420984 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:35.421888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:35.422345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:35.422767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:35.423632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:35.424186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:35.426567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:35.430766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 451/897] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:35.437198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:35.437932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 447/898] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:35.439092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:35.570820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca7df-1b3e-40b0-b43c-12e04e390638 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:35.609936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:35.610682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:35.610964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:35.611259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:35.612708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] RBAC: Adding query filter params (role.id=3aafc58b4d6743218143e36035ba7b4f, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=77bd2bfa6a5343f69db3af35547a261c) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:35.613123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:35.614634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:35.619731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b13fcc-04ab-4de5-bc96-826f0c85f6c0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:35.625634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 452/899] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:35 2022] GET /identity/v3/role_assignments?user.id=77bd2bfa6a5343f69db3af35547a261c&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=3aafc58b4d6743218143e36035ba7b4f => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:35.696566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:35.698376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:35.699183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:35.699992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:35.701787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:35.702719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:35.705886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:35.721209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 448/900] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:35.726679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:35.770437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:35.856250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef67b54-f85e-4eee-b910-00b8a4090913 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:35.915404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e5016-1891-4b72-8bb0-c9942ab8202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:35.916084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5016-1891-4b72-8bb0-c9942ab8202d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:35.916366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5016-1891-4b72-8bb0-c9942ab8202d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:35.916639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5016-1891-4b72-8bb0-c9942ab8202d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:35.917489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e5016-1891-4b72-8bb0-c9942ab8202d None admin] RBAC: Adding query filter params (interface=public, service_id=7fb7cc146e5e4544921e961cd41ec4e9, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:35.917803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e5016-1891-4b72-8bb0-c9942ab8202d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:35.919309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e5016-1891-4b72-8bb0-c9942ab8202d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:35.927034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 453/901] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:42:35 2022] GET /identity/v3/endpoints?service_id=7fb7cc146e5e4544921e961cd41ec4e9&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:35.936056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:36.052619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:36.143271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ffb8c-63ee-41fb-a284-98731b518b40 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:36.210421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51e7fb-fac6-423f-9ab2-57c958959bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:36.211996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e7fb-fac6-423f-9ab2-57c958959bea None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:36.212788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e7fb-fac6-423f-9ab2-57c958959bea None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:36.213571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51e7fb-fac6-423f-9ab2-57c958959bea None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:36.215228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51e7fb-fac6-423f-9ab2-57c958959bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:36.218292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51e7fb-fac6-423f-9ab2-57c958959bea None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:36.231559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 449/902] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:42:35 2022] GET /identity/v3/services => generated 1268 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:42:36.487268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7020de-99f3-4de3-ab38-0d966e9eb69b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:36.487775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7020de-99f3-4de3-ab38-0d966e9eb69b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:36.488166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7020de-99f3-4de3-ab38-0d966e9eb69b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:36.490826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 454/903] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:36.500320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8085754-6f67-4b1c-bc33-0a28ad216b4a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:36.500869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8085754-6f67-4b1c-bc33-0a28ad216b4a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:36.501363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8085754-6f67-4b1c-bc33-0a28ad216b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:36.532093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f8085754-6f67-4b1c-bc33-0a28ad216b4a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:36.534787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f8085754-6f67-4b1c-bc33-0a28ad216b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:36.663352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 450/904] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:36.695875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7020de-99f3-4de3-ab38-0d966e9eb69b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:36.794989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80273a15-fe9a-45e3-86f5-49ec674b23e9 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:36.797820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80273a15-fe9a-45e3-86f5-49ec674b23e9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:36.798141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80273a15-fe9a-45e3-86f5-49ec674b23e9 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:36.804944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 451/905] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:36.822227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4bd9-966d-4a17-b1f6-09b111093250 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:36.826185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4bd9-966d-4a17-b1f6-09b111093250 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:36.826185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4bd9-966d-4a17-b1f6-09b111093250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:36.870682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-eefd4bd9-966d-4a17-b1f6-09b111093250 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:36.873169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-eefd4bd9-966d-4a17-b1f6-09b111093250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:36.944878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7020de-99f3-4de3-ab38-0d966e9eb69b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:36.951708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:37.050131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:37.072101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:37.072996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:37.073280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:37.073608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:37.074339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:37.075822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:37.100333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:37.102157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:37.103866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:37.116874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 452/906] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:37.117168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 455/907] 10.222.0.72 () {56 vars in 1132 bytes} [Thu Jun 16 14:42:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:37.123782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd4bd9-966d-4a17-b1f6-09b111093250 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:37.190728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:37.367250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd4bd9-966d-4a17-b1f6-09b111093250 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:37.488661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295bb7e-f2ae-4a56-bf2f-363f07aab096 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:37.508885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:37.508885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:37.508885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:37.508885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:37.512173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:37.516872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:37.520876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:37.543111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 453/908] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:37.549370 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:37.566736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:37.568175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:37.568994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:37.569560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:37.571243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:37.574468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:37.596931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 456/909] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:42:37 2022] POST /identity/v3/services => generated 241 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:42:37.642799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:37.750808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee2e6-4ef4-4006-81cd-1ed5c9378059 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:37.834749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:37.835936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:37.836427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:37.836953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:37.838339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:37.838885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:37.841670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:37.852764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 454/910] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:37.858980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:37.897549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:38.048058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195adfd-2426-4625-8e8d-ef3e4e49a687 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:38.102321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95474e6d-5aa8-469c-b14a-1722c5c86a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:38.103536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95474e6d-5aa8-469c-b14a-1722c5c86a19 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:38.104244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95474e6d-5aa8-469c-b14a-1722c5c86a19 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:38.104965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95474e6d-5aa8-469c-b14a-1722c5c86a19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:38.106253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95474e6d-5aa8-469c-b14a-1722c5c86a19 None admin] RBAC: Adding query filter params (interface=public, service_id=9a96c71a690947c5b22f23371e384e16, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:38.107003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95474e6d-5aa8-469c-b14a-1722c5c86a19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:38.109895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95474e6d-5aa8-469c-b14a-1722c5c86a19 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:38.120213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 457/911] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:42:37 2022] GET /identity/v3/endpoints?service_id=9a96c71a690947c5b22f23371e384e16&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:38.126936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:38.284859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea049eeb-8604-496e-b38e-1e8366fb0219 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:38.288322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:38.334571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07444042-80c1-4195-91ab-2635ca984839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:38.335262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444042-80c1-4195-91ab-2635ca984839 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:38.335569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444042-80c1-4195-91ab-2635ca984839 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:38.335828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444042-80c1-4195-91ab-2635ca984839 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:38.336517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07444042-80c1-4195-91ab-2635ca984839 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:38.338103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07444042-80c1-4195-91ab-2635ca984839 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:38.345976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 455/912] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:42:38 2022] GET /identity/v3/services => generated 1497 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:42:38.763514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5768db-1b9d-4b05-94dd-2f12b89085b6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:38.764608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5768db-1b9d-4b05-94dd-2f12b89085b6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:38.765148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5768db-1b9d-4b05-94dd-2f12b89085b6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:38.767209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 458/913] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:38.774319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45006355-61dd-43f7-b332-fc8880d0155e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:38.775338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45006355-61dd-43f7-b332-fc8880d0155e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:38.775842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45006355-61dd-43f7-b332-fc8880d0155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:38.796371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-45006355-61dd-43f7-b332-fc8880d0155e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:38.798131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-45006355-61dd-43f7-b332-fc8880d0155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:38.915888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 456/914] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:38.941141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5768db-1b9d-4b05-94dd-2f12b89085b6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:39.054196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:39.076007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5768db-1b9d-4b05-94dd-2f12b89085b6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:39.115416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:39.116615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:39.117137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:39.117626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:39.133528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:39.142864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:39.144204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:42:39.145085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:39.148003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 459/915] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Jun 16 14:42:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:39.154443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45006355-61dd-43f7-b332-fc8880d0155e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:39.313724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:39.320618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45006355-61dd-43f7-b332-fc8880d0155e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:39.378101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:39.378817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:39.379107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:39.379388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:39.380205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:39.380518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:39.382007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:39.394493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 457/916] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Jun 16 14:42:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:39.400986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:39.544992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cf910-4ac7-4a70-9e46-0ceffbd1f343 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:39.587899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77343e71-5c74-47de-b174-50dc3cb3183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:39.589127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343e71-5c74-47de-b174-50dc3cb3183d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:39.589682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343e71-5c74-47de-b174-50dc3cb3183d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:39.590179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77343e71-5c74-47de-b174-50dc3cb3183d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:39.603640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77343e71-5c74-47de-b174-50dc3cb3183d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:39.605636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77343e71-5c74-47de-b174-50dc3cb3183d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:39.617148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-77343e71-5c74-47de-b174-50dc3cb3183d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:39.628178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:39.630054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:39.632419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:39.632419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:39.632419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:39.632419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:39.632419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 460/917] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:39.637997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:39.677613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddef17-f1f8-448c-9e0e-3941105a57ec None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:39.677902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddef17-f1f8-448c-9e0e-3941105a57ec None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:39.678203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddef17-f1f8-448c-9e0e-3941105a57ec None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:39.680014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 461/918] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:39.687155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0643df-f5dc-4131-805b-2baf279f34d8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:39.687448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0643df-f5dc-4131-805b-2baf279f34d8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:39.687725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0643df-f5dc-4131-805b-2baf279f34d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:39.718055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1c0643df-f5dc-4131-805b-2baf279f34d8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:39.720289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1c0643df-f5dc-4131-805b-2baf279f34d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:39.834618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23365-7583-4ad8-8a9f-ea251af0c3ff None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:39.853642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 462/919] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:39.889435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:39.890244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:39.890625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:39.891020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:39.891920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:39.892353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:39.894247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:39.897557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0643df-f5dc-4131-805b-2baf279f34d8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:39.910343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 458/920] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:39.916196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:39.941552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:40.036612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994deedc-fc11-4810-978e-56b621c093ae None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.076495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:40.077358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.077641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.077915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.086178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:40.087968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:40.104195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0643df-f5dc-4131-805b-2baf279f34d8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:40.104380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:40.105392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:40.106362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:40.110428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 459/921] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:42:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:40.111387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:40.173074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:40.175309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.175901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.176486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.177952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:40.180789 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:40.200119 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:40.205196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:40.210772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:40.214648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 14:42:40.218315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:40.223216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 463/922] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:40.225925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:40.236530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a45e2-bf57-4583-823d-263163544d3c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.287369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:40.288351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.288859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.289434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.290690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:40.290973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:40.293041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:40.329315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 460/923] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Jun 16 14:42:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:40.333001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:40.442397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad36aa4-dbd0-4652-8182-e46657ab1629 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.498950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a13a0-f403-4390-8596-cb8b7f29bc0a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.507289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365ce19-bfbb-4646-bf8a-674dad5a29ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:40.507532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365ce19-bfbb-4646-bf8a-674dad5a29ad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.507790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365ce19-bfbb-4646-bf8a-674dad5a29ad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.508067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365ce19-bfbb-4646-bf8a-674dad5a29ad None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.508799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365ce19-bfbb-4646-bf8a-674dad5a29ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:40.509163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365ce19-bfbb-4646-bf8a-674dad5a29ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:40.510566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365ce19-bfbb-4646-bf8a-674dad5a29ad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:40.522482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 464/924] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:40.523833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea003-639c-4c20-89fb-f367c15a0efa None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.524138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea003-639c-4c20-89fb-f367c15a0efa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.524405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea003-639c-4c20-89fb-f367c15a0efa None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.528905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 465/925] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:40.528905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ea003-639c-4c20-89fb-f367c15a0efa None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:40.572953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf6051-4c97-46b3-b7ec-51a3e8ea04f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:40.574321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6051-4c97-46b3-b7ec-51a3e8ea04f4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.574832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6051-4c97-46b3-b7ec-51a3e8ea04f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.575322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6051-4c97-46b3-b7ec-51a3e8ea04f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.588286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:40.595128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf6051-4c97-46b3-b7ec-51a3e8ea04f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:40.597966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf6051-4c97-46b3-b7ec-51a3e8ea04f4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:40.618317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d1bf6051-4c97-46b3-b7ec-51a3e8ea04f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:40.628482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:40.629624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:40.630537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:40.630537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:40.630537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:40.630537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:40.630537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 461/926] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:40.631114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71011f82-d7ee-4636-b963-554eca252254 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.631334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71011f82-d7ee-4636-b963-554eca252254 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.631611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71011f82-d7ee-4636-b963-554eca252254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.653730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-71011f82-d7ee-4636-b963-554eca252254 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:40.654987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-71011f82-d7ee-4636-b963-554eca252254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:40.709650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ea003-639c-4c20-89fb-f367c15a0efa None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.751827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 462/927] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:40.752979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71011f82-d7ee-4636-b963-554eca252254 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:40.761343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:40.762454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:40.762742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:40.763024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:40.763767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:40.764071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:40.765507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:40.777110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 466/928] 10.222.0.72 () {56 vars in 1114 bytes} [Thu Jun 16 14:42:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:40.778183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:40.937436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71011f82-d7ee-4636-b963-554eca252254 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.954911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd745bd-8ddc-4a5e-9ebb-8eaa3ac979fd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:40.980405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:40.988720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:41.004328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:41.005075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.005542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.005822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.006721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:41.007105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:41.008659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:41.012059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:41.013268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.013778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.014282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.015781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:41.019113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:41.025110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 463/929] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:41.026319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:41.060919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 467/930] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:42:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:41.062335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:41.279451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41521942-237a-469d-8849-07bbceed6ea0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:41.289132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7f0b3-de8a-43de-b856-a24319bc55d3 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:41.356900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:41.358814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.359425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.359935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.365689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:41.365689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:41.383518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:41.385284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.385940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.386600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:41.391489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:41.393313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:41.395407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 464/931] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:41.405965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:41.406365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:41.408999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:41.440879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:41.442434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:41.444031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:41.444792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 468/932] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:41.455323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:41.476428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:41.531980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:41.576820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40524f06-261d-4209-995b-d888cfc95036 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:41.627409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24341e-195c-4247-a068-3a5fdf940594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:41.628711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24341e-195c-4247-a068-3a5fdf940594 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.629034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24341e-195c-4247-a068-3a5fdf940594 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.629319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24341e-195c-4247-a068-3a5fdf940594 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.630177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24341e-195c-4247-a068-3a5fdf940594 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:41.630493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24341e-195c-4247-a068-3a5fdf940594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:41.632044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24341e-195c-4247-a068-3a5fdf940594 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:41.643392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 465/933] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:41.643392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847dfc6-4dcb-4de6-8c5f-8ed8e70d98fc None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.643392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847dfc6-4dcb-4de6-8c5f-8ed8e70d98fc None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.643392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847dfc6-4dcb-4de6-8c5f-8ed8e70d98fc None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.643392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 466/934] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:41.648249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfba0b-472d-4fe9-87d2-de3879161159 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.648524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfba0b-472d-4fe9-87d2-de3879161159 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.648971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfba0b-472d-4fe9-87d2-de3879161159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.654451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86932a0-22a1-4981-9065-cf65531b3e24 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:41.667794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-99dfba0b-472d-4fe9-87d2-de3879161159 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:41.669135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-99dfba0b-472d-4fe9-87d2-de3879161159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:41.726446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:41.727610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:41.728115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:41.728566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:41.730125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:41.730623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:41.733297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:41.757040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 469/935] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:41.761162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:41.804980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 467/936] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:41.804980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfba0b-472d-4fe9-87d2-de3879161159 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:41.877892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:41.966423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:41.972914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfba0b-472d-4fe9-87d2-de3879161159 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:41.979926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab09a-13af-49bf-a9d5-b73a91a02d4d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:42.036441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:42.037956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:42.038313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:42.038785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/77bd2bfa6a5343f69db3af35547a261c/roles/3aafc58b4d6743218143e36035ba7b4f` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:42.042422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=77bd2bfa6a5343f69db3af35547a261c, role_id=3aafc58b4d6743218143e36035ba7b4f)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:42.045063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:42.048586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:42.050552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:42.051261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:42.051668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:42.052992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:42.053533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:42.056918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:42.064644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 470/937] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:42.065770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:42.077526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 468/938] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:42:41 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/77bd2bfa6a5343f69db3af35547a261c/roles/3aafc58b4d6743218143e36035ba7b4f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:42:42.079424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:42.265282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb9e5e-319e-4c4e-bbf7-314892a8d35a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:42.298004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:42.306885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630089b7-bd73-43d7-8151-90ea8c1f803f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:42.332398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:42.333850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:42.334304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:42.334732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:42.335865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:42.338271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:42.361727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:42.363974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:42.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 471/939] 10.222.0.72 () {56 vars in 1132 bytes} [Thu Jun 16 14:42:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:42.375900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:42.400392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:42.402187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:42.402997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:42.403755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:42.405695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] RBAC: Adding query filter params (interface=public, service_id=4bc989dc9a9c4acba88d2f9b1ad0e979, region_id=RegionOne) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:42.406590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:42.409514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:42.425018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 469/940] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:42:42 2022] GET /identity/v3/endpoints?service_id=4bc989dc9a9c4acba88d2f9b1ad0e979&interface=public®ion_id=RegionOne => generated 194 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:42.430913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:42.431616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:42.599601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf640-bc33-4bbf-96de-2039522b2df9 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:42.632670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b89d3-3a4c-499b-b786-5add5da56ab8 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:42.670126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:42.671309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:42.671809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:42.672283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:42.673676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:42.674296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:42.677778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:42.687899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 472/941] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:42.694913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:42.706832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:42.708436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:42.709271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:42.710009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:42.711588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:42.714768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:42.737261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 470/942] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:42:42 2022] GET /identity/v3/services => generated 1497 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:42:42.913448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:42.921974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86af70-8125-48d0-b157-d761d3f5f785 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:42.995660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94245336-ea6b-41ad-be31-2834fe828aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:42.996883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94245336-ea6b-41ad-be31-2834fe828aca None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:42.997344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94245336-ea6b-41ad-be31-2834fe828aca None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:42.997791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94245336-ea6b-41ad-be31-2834fe828aca None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:42.999205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94245336-ea6b-41ad-be31-2834fe828aca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:42.999750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94245336-ea6b-41ad-be31-2834fe828aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:43.003065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94245336-ea6b-41ad-be31-2834fe828aca None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:43.013706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 473/943] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:43.021446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:43.077890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:43.228670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0391653-cff7-4d9e-8f05-e6d2c917fd6c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:43.296290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee1771-d310-4ba1-87c9-2ecb06fd4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:43.297517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee1771-d310-4ba1-87c9-2ecb06fd4af1 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:43.298000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee1771-d310-4ba1-87c9-2ecb06fd4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:43.298483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee1771-d310-4ba1-87c9-2ecb06fd4af1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:43.299937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee1771-d310-4ba1-87c9-2ecb06fd4af1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:43.302559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee1771-d310-4ba1-87c9-2ecb06fd4af1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:43.349535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 471/944] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:42:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:42:44.291039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43704-c657-4bf0-af98-df13e0ad162e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:44.292016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43704-c657-4bf0-af98-df13e0ad162e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:44.292465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43704-c657-4bf0-af98-df13e0ad162e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:44.294412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 474/945] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:44.301187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd158d9-6248-41ae-bebe-2a332d7dbb05 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:44.302157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd158d9-6248-41ae-bebe-2a332d7dbb05 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:44.302606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd158d9-6248-41ae-bebe-2a332d7dbb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:44.329672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0dd158d9-6248-41ae-bebe-2a332d7dbb05 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:44.331880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0dd158d9-6248-41ae-bebe-2a332d7dbb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:44.484524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 472/946] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:44.522539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff43704-c657-4bf0-af98-df13e0ad162e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:44.743772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff43704-c657-4bf0-af98-df13e0ad162e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:44.816645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:44.824910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:44.824910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:44.824910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:44.824910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:44.824910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:44.825636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING keystone.server.flask.common [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 14:42:44.860511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.sql.core [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 14:42:44.860511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 14:42:44.860511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [parameters: {'id': 'b1570bd163e4486f9442ce177aca9ccf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 14:42:44.860511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86693) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1570bd163e4486f9442ce177aca9ccf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1570bd163e4486f9442ce177aca9ccf. Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:42:44.894077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 14:42:44.895684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application yield resource Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise self.value Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application session.commit() Jun 16 14:42:44.898341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.flush() Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application compat.raise_( Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 14:42:44.900318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 14:42:44.902256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:42:44.903952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [parameters: {'id': 'b1570bd163e4486f9442ce177aca9ccf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:44.911336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:44.914542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b1570bd163e4486f9442ce177aca9ccf. Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:44.918018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 475/947] 10.222.0.72 () {60 vars in 1140 bytes} [Thu Jun 16 14:42:44 2022] POST /identity/v3/projects => generated 262 bytes in 376 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:44.925513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd158d9-6248-41ae-bebe-2a332d7dbb05 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:44.957292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:45.141169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd158d9-6248-41ae-bebe-2a332d7dbb05 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:45.184599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3e4c0-27e6-4a45-8fec-d344f9176cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:45.185473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e4c0-27e6-4a45-8fec-d344f9176cfc None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:45.185760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e4c0-27e6-4a45-8fec-d344f9176cfc None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:45.186021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e4c0-27e6-4a45-8fec-d344f9176cfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:45.194955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3e4c0-27e6-4a45-8fec-d344f9176cfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:45.196484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3e4c0-27e6-4a45-8fec-d344f9176cfc None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-bbe3e4c0-27e6-4a45-8fec-d344f9176cfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:45.212421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:45.215547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:45.218048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 473/948] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:45.228510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:45.240592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:45.366110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68072728-34f1-4e64-8529-a0bff1315d21 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:45.408426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:45.409252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:45.409646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:45.410098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:45.411066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:45.411484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:45.413396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:45.426446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41c4b-6d02-4ea5-bc1a-238644dd0681 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:45.427200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41c4b-6d02-4ea5-bc1a-238644dd0681 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:45.427480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff41c4b-6d02-4ea5-bc1a-238644dd0681 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:45.436895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 474/949] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:45.439435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67675a2-9a2f-404c-bd80-bf3964490445 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:45.440485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67675a2-9a2f-404c-bd80-bf3964490445 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:45.441234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67675a2-9a2f-404c-bd80-bf3964490445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:45.446713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 476/950] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:45.468495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f67675a2-9a2f-404c-bd80-bf3964490445 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:45.470025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f67675a2-9a2f-404c-bd80-bf3964490445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:45.568939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 475/951] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:45.569409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:45.610760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:45.702564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:45.768007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c74bb7-6ec3-41c5-b7d5-0d69b9b6111e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:45.844054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:45.845796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:45.847030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:45.847690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:45.868437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:42:45.892360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:45.894690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:42:45.896457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:45.900639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 477/952] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Jun 16 14:42:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:45.910528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67675a2-9a2f-404c-bd80-bf3964490445 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:46.045535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67675a2-9a2f-404c-bd80-bf3964490445 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:46.088789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aed22b-0108-4082-a720-586323254d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:46.089723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed22b-0108-4082-a720-586323254d55 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.090110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed22b-0108-4082-a720-586323254d55 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.090490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed22b-0108-4082-a720-586323254d55 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.091389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aed22b-0108-4082-a720-586323254d55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:46.091811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aed22b-0108-4082-a720-586323254d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:46.093422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aed22b-0108-4082-a720-586323254d55 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:46.104630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 476/953] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Jun 16 14:42:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:46.110254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:46.278908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fa7fa-f0f9-489d-befb-862ad9556ca8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:46.332987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:46.350289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:46.351931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.352710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.353254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.367622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d54de6-3762-4c2b-acdc-032199d34add None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.367935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d54de6-3762-4c2b-acdc-032199d34add None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.368211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d54de6-3762-4c2b-acdc-032199d34add None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.375185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:46.377889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 477/954] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:46.379507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:46.379937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95241f7-75fe-4c4c-80c5-dea7cdb48011 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.380362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95241f7-75fe-4c4c-80c5-dea7cdb48011 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.380869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95241f7-75fe-4c4c-80c5-dea7cdb48011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.409020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c95241f7-75fe-4c4c-80c5-dea7cdb48011 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:46.411017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c95241f7-75fe-4c4c-80c5-dea7cdb48011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:46.414534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:46.416177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:46.422460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:46.422460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:46.422460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:46.422460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:46.422460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 478/955] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:46.427259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:46.540351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:46.574985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 478/956] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:46.617505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95241f7-75fe-4c4c-80c5-dea7cdb48011 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:46.632695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c83e63-6ec9-4b9b-b1ce-394d1552ab15 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:46.697415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe7306-989c-4611-8d7e-992af020e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:46.698316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe7306-989c-4611-8d7e-992af020e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.699120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe7306-989c-4611-8d7e-992af020e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.699299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe7306-989c-4611-8d7e-992af020e0f2 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.700311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe7306-989c-4611-8d7e-992af020e0f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:46.700773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe7306-989c-4611-8d7e-992af020e0f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:46.702382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe7306-989c-4611-8d7e-992af020e0f2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:46.718213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 479/957] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:46.724310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c0f19-7a29-4db1-b998-67ae4ff4a21f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.724630 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c0f19-7a29-4db1-b998-67ae4ff4a21f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.724917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c0f19-7a29-4db1-b998-67ae4ff4a21f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.726719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 480/958] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:46.727196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c0f19-7a29-4db1-b998-67ae4ff4a21f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:46.789201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:46.807841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95241f7-75fe-4c4c-80c5-dea7cdb48011 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:46.882534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9171189-5893-4c05-b828-9f6c7998ec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:46.884293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171189-5893-4c05-b828-9f6c7998ec43 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.884293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171189-5893-4c05-b828-9f6c7998ec43 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.884555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9171189-5893-4c05-b828-9f6c7998ec43 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.885695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9171189-5893-4c05-b828-9f6c7998ec43 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:46.889193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9171189-5893-4c05-b828-9f6c7998ec43 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:46.893645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c0f19-7a29-4db1-b998-67ae4ff4a21f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-a9171189-5893-4c05-b828-9f6c7998ec43 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:46.915322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:46.917368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:46.919265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 479/959] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:46.920884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157bb6-7ff5-43b6-b889-d30c3fdc8ad1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.920884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157bb6-7ff5-43b6-b889-d30c3fdc8ad1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.921124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157bb6-7ff5-43b6-b889-d30c3fdc8ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.949188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:46.950369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:46.950804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:46.951210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:46.951667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-9f157bb6-7ff5-43b6-b889-d30c3fdc8ad1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:46.954016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-9f157bb6-7ff5-43b6-b889-d30c3fdc8ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:46.962829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:46.964553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:46.986327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:46.988491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 481/960] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:42:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:46.996822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:47.075056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:47.126772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 480/961] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:47.126772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f157bb6-7ff5-43b6-b889-d30c3fdc8ad1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:47.187932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:47.195754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736203d8-070d-47c8-b419-9214474509e6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:47.259430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:47.260810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:47.261378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:47.261950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:47.263237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:47.263802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:47.266518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:47.283439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 482/962] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:47.283439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:47.306144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f157bb6-7ff5-43b6-b889-d30c3fdc8ad1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:47.354057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:47.354754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:47.355033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:47.355313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:47.356242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:47.356559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:47.358148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:47.384037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 481/963] 10.222.0.72 () {56 vars in 1149 bytes} [Thu Jun 16 14:42:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:47.389412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:47.453664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:47.474184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee634b-604e-46ba-afa9-f624f34e164b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:47.551456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:47.552734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:47.553224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:47.553659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] PATH_INFO: `/v3/services/compute` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:47.555079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:47.557471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:47.578539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:47.580568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:47.582404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 483/964] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:47.583978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:47.591504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fe3d7-81bd-4319-ba14-713860940104 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:47.646036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:47.679191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:47.680728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:47.681270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:47.681761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:47.683121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:47.683665 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:47.686372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:47.705177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 482/965] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:47.710874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:47.807996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27c3cc-3b93-421f-81aa-8dc2126a42ca None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:47.899723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:47.900826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:47.901323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:47.901773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:47.951427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f66fba-d0c0-4842-a149-6a81c6229f6c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:47.964613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:48.011162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:48.012363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:48.012889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:48.013391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:48.014733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:48.015293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:48.018905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:48.026694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:48.034764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 483/966] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:48.034764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:48.035023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:48.081054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:48.201315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754a4c9-7082-4425-93d3-3845fff63cb3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:48.204005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 484/967] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:42:47 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:42:48.206227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:48.269187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:48.269814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:48.270095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:48.270383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:48.271134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:48.272559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:48.308880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 484/968] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:42:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:48.308880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:48.354410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:48.453424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a2078-bcb6-4b15-8e8c-9587d8a36035 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:48.494855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:48.521665 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7323739-c56b-44e4-8410-6407a074140d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:48.524439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:48.525718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:48.526449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:48.527179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:48.528818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:48.529454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:48.531117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:48.545473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 485/969] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:48.599993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:48.601465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:48.601976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:48.602477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:48.625810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:48.628978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:42:48.666768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:48.668588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:48.670427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:42:48.670427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:42:48.670427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:42:48.670427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:48.670427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 485/970] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:48.679549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:48.869591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7a20d-fe75-4bc0-8edc-8745ecceffa5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:48.930419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9292c-a8f3-46c0-b6ec-abd7ce10e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:48.931137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9292c-a8f3-46c0-b6ec-abd7ce10e947 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:48.931424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9292c-a8f3-46c0-b6ec-abd7ce10e947 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:48.931707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9292c-a8f3-46c0-b6ec-abd7ce10e947 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:48.932633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9292c-a8f3-46c0-b6ec-abd7ce10e947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:48.933168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9292c-a8f3-46c0-b6ec-abd7ce10e947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:48.934929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9292c-a8f3-46c0-b6ec-abd7ce10e947 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:48.950386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 486/971] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:48.957096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:49.010912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387eb3e9-70ef-4251-a9f8-c371949eb733 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:49.011560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387eb3e9-70ef-4251-a9f8-c371949eb733 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:49.011844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387eb3e9-70ef-4251-a9f8-c371949eb733 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:49.014787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 487/972] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:49.024051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7126-e30b-4d17-9daf-2f5ffe3205ec None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:49.024510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7126-e30b-4d17-9daf-2f5ffe3205ec None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:49.025043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7126-e30b-4d17-9daf-2f5ffe3205ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:49.057257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2f0b7126-e30b-4d17-9daf-2f5ffe3205ec None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:49.060880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2f0b7126-e30b-4d17-9daf-2f5ffe3205ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:49.114555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:49.155570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:49.163518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b3a82-e761-4df6-ba6f-2c700393de17 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:49.211267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 488/973] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:49.231075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:49.232402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:49.233010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:49.233506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:49.243319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b7126-e30b-4d17-9daf-2f5ffe3205ec None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:49.247116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:49.251751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:42:49.283008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:49.284551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:42:49.285539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:49.288666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 486/974] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:49.299806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:49.432578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b7126-e30b-4d17-9daf-2f5ffe3205ec None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:49.487994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985b9be-146e-492c-8ad9-2bf2dcf184a4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:49.497533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:49.499807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:49.500596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:49.501241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:49.502269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:49.504033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:49.522357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.sql.core [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 14:42:49.522357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 14:42:49.522357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [parameters: {'user_id': '5a6d71d9999142969545b4faf9d98f89', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 14:42:49.522357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86693) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() Jun 16 14:42:49.540532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:42:49.542057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application yield resource Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application next(self.gen) Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise self.value Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application session.commit() Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 14:42:49.543046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.session.flush() Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application compat.raise_( Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 14:42:49.543949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 14:42:49.545695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application conn.query(q) Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application result.read() Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 14:42:49.547382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [parameters: {'user_id': '5a6d71d9999142969545b4faf9d98f89', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:49.548531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 14:42:49.551838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 489/975] 10.222.0.72 () {60 vars in 1131 bytes} [Thu Jun 16 14:42:49 2022] POST /identity/v3/users => generated 160 bytes in 301 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:49.554943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:49.565323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:49.566747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:49.567433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:49.568099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:49.570979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:49.571868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:49.575600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:49.579923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:49.607576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 487/976] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:49.615377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:49.632982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:49.747524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e7a42-60f5-4aac-8d8d-c71290c725aa None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:49.802270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:49.803877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:49.803877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:49.804209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:49.811623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ea070-a871-484c-b8a8-126e061b8b4a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:49.816117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:49.818052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:42:49.848587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:49.849936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 14:42:49.851101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:49.862240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 490/977] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:42:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:49.864413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:49.907274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206af03-852e-4741-ba16-b6a17ce9bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:49.909484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206af03-852e-4741-ba16-b6a17ce9bfaa None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:49.910258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206af03-852e-4741-ba16-b6a17ce9bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:49.910963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206af03-852e-4741-ba16-b6a17ce9bfaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:49.913027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206af03-852e-4741-ba16-b6a17ce9bfaa None admin] RBAC: Adding query filter params (role.id=3aafc58b4d6743218143e36035ba7b4f, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=77bd2bfa6a5343f69db3af35547a261c) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:49.913859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206af03-852e-4741-ba16-b6a17ce9bfaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:49.916991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206af03-852e-4741-ba16-b6a17ce9bfaa None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:49.931204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 488/978] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:42:49 2022] GET /identity/v3/role_assignments?user.id=77bd2bfa6a5343f69db3af35547a261c&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=3aafc58b4d6743218143e36035ba7b4f => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:50.115086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:50.123121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d9ced-f4fd-4758-b226-3b63cfe4f7c6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:50.200004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66223e-b960-4e7f-9550-5af32dff48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:50.201061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66223e-b960-4e7f-9550-5af32dff48de None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:50.201464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66223e-b960-4e7f-9550-5af32dff48de None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:50.201850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66223e-b960-4e7f-9550-5af32dff48de None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:50.203243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66223e-b960-4e7f-9550-5af32dff48de None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:50.203686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66223e-b960-4e7f-9550-5af32dff48de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:50.206507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66223e-b960-4e7f-9550-5af32dff48de None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:50.287558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 491/979] 10.222.0.72 () {56 vars in 1101 bytes} [Thu Jun 16 14:42:49 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:51.625949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a346-15a7-4eab-b21a-313617bf2fee None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:51.626573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a346-15a7-4eab-b21a-313617bf2fee None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:51.626945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a346-15a7-4eab-b21a-313617bf2fee None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:51.629958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 489/980] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:51.636431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0ee7-891a-4da3-ad1b-976b6aa9872b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:51.637281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0ee7-891a-4da3-ad1b-976b6aa9872b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:51.637281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a0ee7-891a-4da3-ad1b-976b6aa9872b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:51.665949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-872a0ee7-891a-4da3-ad1b-976b6aa9872b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:51.668635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-872a0ee7-891a-4da3-ad1b-976b6aa9872b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:51.798655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca988d-32b9-4ad5-9699-d5dfbc3a5187 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:51.799161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca988d-32b9-4ad5-9699-d5dfbc3a5187 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:51.799653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca988d-32b9-4ad5-9699-d5dfbc3a5187 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:51.803115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 490/981] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:51.812203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e75b5-a1c3-4d70-a900-ff18c3af79b5 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:51.812711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e75b5-a1c3-4d70-a900-ff18c3af79b5 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:51.813264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e75b5-a1c3-4d70-a900-ff18c3af79b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:51.822012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 492/982] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:51.843248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-085e75b5-a1c3-4d70-a900-ff18c3af79b5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:51.845373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-085e75b5-a1c3-4d70-a900-ff18c3af79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:51.852818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a0ee7-891a-4da3-ad1b-976b6aa9872b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:51.961622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:51.969832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:51.989230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 491/983] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:51.990675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a0ee7-891a-4da3-ad1b-976b6aa9872b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:52.025711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e75b5-a1c3-4d70-a900-ff18c3af79b5 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:52.063641 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:52.064681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:52.065117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:52.065571 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:52.066829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:52.069069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:52.097113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 493/984] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:42:51 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:52.233196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e75b5-a1c3-4d70-a900-ff18c3af79b5 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:52.328825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:52.331123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:52.331488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:52.331938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:52.333238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:52.335898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:52.364005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:52.373838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:52.376256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 492/985] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:42:52.384481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:52.593980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fcfc4-50f3-4962-a651-414aeae9a33e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:52.667967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:52.669203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:52.669611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:52.670093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:52.671405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:52.671868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:52.674810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:52.683797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 494/986] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:52.694592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:52.765582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:52.845022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d1a81-4cfb-4f42-b1a3-eb303676d2fd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:52.922931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bca90-ccbc-4d45-815a-7a54da44078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:52.924685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bca90-ccbc-4d45-815a-7a54da44078e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:52.925212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bca90-ccbc-4d45-815a-7a54da44078e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:52.925820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bca90-ccbc-4d45-815a-7a54da44078e None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:52.927097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bca90-ccbc-4d45-815a-7a54da44078e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:52.927637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bca90-ccbc-4d45-815a-7a54da44078e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:52.930381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bca90-ccbc-4d45-815a-7a54da44078e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:52.941707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 493/987] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:52.951030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:52.975085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:53.028566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61845f76-e44c-41a0-8630-b94f40121494 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:53.030106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61845f76-e44c-41a0-8630-b94f40121494 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:53.030106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61845f76-e44c-41a0-8630-b94f40121494 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:53.032906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 494/988] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:53.042129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d7f6-a242-474a-a176-526654f81eed None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:53.042635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d7f6-a242-474a-a176-526654f81eed None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:53.043142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d7f6-a242-474a-a176-526654f81eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:53.077266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a689d7f6-a242-474a-a176-526654f81eed None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:53.079448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a689d7f6-a242-474a-a176-526654f81eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:53.145550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642edb7-9867-4c65-aef9-ba7e02923cec None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:53.196073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:53.197525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:53.197869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:53.198191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:53.199306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] RBAC: Adding query filter params (interface=admin, service_id=4bc989dc9a9c4acba88d2f9b1ad0e979, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:53.199722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:53.201010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:53.209023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 495/989] 10.222.0.72 () {56 vars in 1249 bytes} [Thu Jun 16 14:42:52 2022] GET /identity/v3/endpoints?service_id=4bc989dc9a9c4acba88d2f9b1ad0e979&interface=admin®ion_id=RegionOne => generated 193 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:53.216467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:53.226579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 495/990] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:53.269410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689d7f6-a242-474a-a176-526654f81eed None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:53.368210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:53.371249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:53.421682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571e5d-ca97-48eb-bf18-e06ad2153d31 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:53.486156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689d7f6-a242-474a-a176-526654f81eed None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:53.490902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:53.493467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:53.493927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:53.494330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:53.495423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:53.498158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:53.513923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 496/991] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:42:53 2022] GET /identity/v3/services => generated 1727 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:42:53.567240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:53.568345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:53.568716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:53.569036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] PATH_INFO: `/v3/services/image` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:53.569760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:53.571247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:53.584939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:53.586355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:53.590617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 496/992] 10.222.0.72 () {56 vars in 1105 bytes} [Thu Jun 16 14:42:53 2022] GET /identity/v3/services/image => generated 86 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:53.597548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:53.814302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa0d7d-36df-4017-a595-f0ed44a9a80e None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:53.900312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe28fe-04e8-4b87-8252-2c3e9ce3c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:53.902112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe28fe-04e8-4b87-8252-2c3e9ce3c159 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:53.903168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe28fe-04e8-4b87-8252-2c3e9ce3c159 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:53.903902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe28fe-04e8-4b87-8252-2c3e9ce3c159 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:53.905659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe28fe-04e8-4b87-8252-2c3e9ce3c159 None admin] RBAC: Adding query filter params (name=image) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:53.906543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe28fe-04e8-4b87-8252-2c3e9ce3c159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:53.909545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe28fe-04e8-4b87-8252-2c3e9ce3c159 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:53.921135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 497/993] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:53.930953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:54.145167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415916f-7501-417c-9b56-3bf5d4c88150 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:54.154580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:54.189350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80999a6-8462-4a9a-b2dc-9890a1448d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:54.190676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80999a6-8462-4a9a-b2dc-9890a1448d2d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:54.190963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80999a6-8462-4a9a-b2dc-9890a1448d2d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:54.191226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80999a6-8462-4a9a-b2dc-9890a1448d2d None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:54.192013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80999a6-8462-4a9a-b2dc-9890a1448d2d None admin] RBAC: Adding query filter params (type=image) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:54.192324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80999a6-8462-4a9a-b2dc-9890a1448d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:54.193783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80999a6-8462-4a9a-b2dc-9890a1448d2d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:54.204106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 497/994] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:42:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:55.361080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1c65-6a8e-4641-bd02-7233fbaa8c2c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:55.362373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1c65-6a8e-4641-bd02-7233fbaa8c2c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:55.362710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a1c65-6a8e-4641-bd02-7233fbaa8c2c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:55.368753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 498/995] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:55.378182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec5180-0099-43e0-81b5-ca31c41f1937 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:55.379039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec5180-0099-43e0-81b5-ca31c41f1937 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:55.379183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec5180-0099-43e0-81b5-ca31c41f1937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:55.410802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0dec5180-0099-43e0-81b5-ca31c41f1937 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:55.412789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0dec5180-0099-43e0-81b5-ca31c41f1937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:55.589367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 498/996] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:42:55.638471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a1c65-6a8e-4641-bd02-7233fbaa8c2c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:55.644989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:55.871241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a1c65-6a8e-4641-bd02-7233fbaa8c2c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:55.926911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:55.927635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:55.927916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:55.928285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:55.929159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:55.930766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:55.946235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:55.956065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:42:55.957072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 499/997] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:42:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:55.961601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec5180-0099-43e0-81b5-ca31c41f1937 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:56.118490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:56.127097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec5180-0099-43e0-81b5-ca31c41f1937 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:56.203602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:56.203602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:56.203602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:56.204131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:56.205307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] RBAC: Adding query filter params (name=compute) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:56.205871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:56.208537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:56.219435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 499/998] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:56.230295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:56.378088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:56.447629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46662753-5b45-4eaa-a700-3025803cd259 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:56.493570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:56.494162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:56.494450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:56.494705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:56.495465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:56.495764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:56.497497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:56.504659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 500/999] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:42:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:56.511361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:56.731676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c140d7-19e2-4a4f-a533-797033002ccf None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:56.807956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0960c2-ea0d-4856-ba2c-e423e18cc285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:56.812790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0960c2-ea0d-4856-ba2c-e423e18cc285 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:56.812790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0960c2-ea0d-4856-ba2c-e423e18cc285 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:56.812790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0960c2-ea0d-4856-ba2c-e423e18cc285 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:56.812790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0960c2-ea0d-4856-ba2c-e423e18cc285 None admin] RBAC: Adding query filter params (interface=public, service_id=89d2ed8f6c704981b303bdf5a82dee1f, region_id=RegionOne) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:56.812790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0960c2-ea0d-4856-ba2c-e423e18cc285 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:56.818215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0960c2-ea0d-4856-ba2c-e423e18cc285 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:56.832260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 500/1000] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:42:56 2022] GET /identity/v3/endpoints?service_id=89d2ed8f6c704981b303bdf5a82dee1f&interface=public®ion_id=RegionOne => generated 194 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:42:56.834665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:57.006688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227988c-f5b7-43d6-9ef4-428ef7229255 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.007124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227988c-f5b7-43d6-9ef4-428ef7229255 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.007384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227988c-f5b7-43d6-9ef4-428ef7229255 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.009195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 501/1001] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:57.015305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c6b5-1217-4165-823d-45f855e04923 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.015602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c6b5-1217-4165-823d-45f855e04923 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.015869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c6b5-1217-4165-823d-45f855e04923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.037131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f071c6b5-1217-4165-823d-45f855e04923 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:57.038455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f071c6b5-1217-4165-823d-45f855e04923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:57.063721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0e1b9-a012-4bc6-9ac2-fab2df088ae7 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:57.069641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:57.125030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:57.136958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdee75-d581-433f-b201-48576981aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:57.138254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdee75-d581-433f-b201-48576981aa8b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.138737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdee75-d581-433f-b201-48576981aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.139237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdee75-d581-433f-b201-48576981aa8b None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.140293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 502/1002] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:57.140726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdee75-d581-433f-b201-48576981aa8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:57.144427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdee75-d581-433f-b201-48576981aa8b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:57.158233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 501/1003] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:42:56 2022] GET /identity/v3/services => generated 1727 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:42:57.167996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c6b5-1217-4165-823d-45f855e04923 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:57.229275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9ff62-2734-4980-b203-dc072ca18c0f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.229795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9ff62-2734-4980-b203-dc072ca18c0f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.230233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9ff62-2734-4980-b203-dc072ca18c0f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.233172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 502/1004] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:42:57.242267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e3ae-1fbf-403c-99bd-93f9d20afdb9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.242771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e3ae-1fbf-403c-99bd-93f9d20afdb9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.243266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e3ae-1fbf-403c-99bd-93f9d20afdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.273844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-82c7e3ae-1fbf-403c-99bd-93f9d20afdb9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:57.275982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-82c7e3ae-1fbf-403c-99bd-93f9d20afdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:42:57.372365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c6b5-1217-4165-823d-45f855e04923 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:57.443409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 503/1005] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:42:57.447964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:57.450103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.450103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.450428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.451603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:57.454323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:42:57.477695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:42:57.480288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:42:57.483689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 503/1006] 10.222.0.72 () {56 vars in 1126 bytes} [Thu Jun 16 14:42:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:42:57.486671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7e3ae-1fbf-403c-99bd-93f9d20afdb9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:57.490048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:57.541510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:57.678691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7e3ae-1fbf-403c-99bd-93f9d20afdb9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:57.710170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02a804-51de-4d5a-995a-5b20fa7f81d4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:57.732262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:57.754199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:57.755471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.755830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.756342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.757168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:57.759216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:57.783009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 504/1007] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:42:57 2022] POST /identity/v3/services => generated 242 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:42:57.783268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:57.783268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:57.783723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:57.784193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:57.785529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:57.786116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:57.788427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:57.795623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 504/1008] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:57.802420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:58.034123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97907b21-fe10-44dc-97c9-aa2bec6d7ddd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:58.102665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1baf53-ded3-443a-afc2-a5017b92347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:58.103564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1baf53-ded3-443a-afc2-a5017b92347c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:58.103997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1baf53-ded3-443a-afc2-a5017b92347c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:58.104446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1baf53-ded3-443a-afc2-a5017b92347c None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:58.105528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1baf53-ded3-443a-afc2-a5017b92347c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:42:58.106000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1baf53-ded3-443a-afc2-a5017b92347c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:58.107617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1baf53-ded3-443a-afc2-a5017b92347c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:58.115156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 505/1009] 10.222.0.72 () {56 vars in 1122 bytes} [Thu Jun 16 14:42:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:42:58.121371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:42:58.240073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:42:58.287890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203043-3623-4ceb-9f13-1874ea9bef93 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:42:58.342596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704f711-2558-4807-bcb1-1333febaa28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:42:58.343661 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704f711-2558-4807-bcb1-1333febaa28e None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:58.344113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704f711-2558-4807-bcb1-1333febaa28e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:58.344546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704f711-2558-4807-bcb1-1333febaa28e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:58.346565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704f711-2558-4807-bcb1-1333febaa28e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:42:58.349375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704f711-2558-4807-bcb1-1333febaa28e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:42:58.387640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 505/1010] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:42:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:42:59.910804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ac71c-1b7d-40dc-baf5-8d7eb5201cb8 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:59.911656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ac71c-1b7d-40dc-baf5-8d7eb5201cb8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:59.911810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ac71c-1b7d-40dc-baf5-8d7eb5201cb8 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:59.913601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 506/1011] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:42:59.919750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e11e44-c444-49f5-8aaf-841d7cab67a9 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:42:59.920585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e11e44-c444-49f5-8aaf-841d7cab67a9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:42:59.920758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e11e44-c444-49f5-8aaf-841d7cab67a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:42:59.952933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-52e11e44-c444-49f5-8aaf-841d7cab67a9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:42:59.954889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-52e11e44-c444-49f5-8aaf-841d7cab67a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:00.112694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 506/1012] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:42:59 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:00.150906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ac71c-1b7d-40dc-baf5-8d7eb5201cb8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:00.255945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:00.375316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ac71c-1b7d-40dc-baf5-8d7eb5201cb8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:00.453341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:00.454710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:00.455770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:00.456228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] PATH_INFO: `/v3/services/compute` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:00.457587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:00.460340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:00.484222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:00.486112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:00.488748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 507/1013] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:43:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:00.497227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e11e44-c444-49f5-8aaf-841d7cab67a9 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:00.609881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:00.681289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e11e44-c444-49f5-8aaf-841d7cab67a9 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:00.753078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f98e7e-cf89-40a5-bf05-70cf6be4a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:00.754175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98e7e-cf89-40a5-bf05-70cf6be4a474 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:00.754666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98e7e-cf89-40a5-bf05-70cf6be4a474 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:00.755251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98e7e-cf89-40a5-bf05-70cf6be4a474 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:00.756621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f98e7e-cf89-40a5-bf05-70cf6be4a474 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:00.757214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f98e7e-cf89-40a5-bf05-70cf6be4a474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:00.759663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f98e7e-cf89-40a5-bf05-70cf6be4a474 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:00.775660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 507/1014] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:43:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:00.777684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:00.941099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ebc4b-625c-4f2b-999e-69c7ca751f19 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:00.941873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ebc4b-625c-4f2b-999e-69c7ca751f19 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:00.942164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ebc4b-625c-4f2b-999e-69c7ca751f19 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:00.945289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 508/1015] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:00.954350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e638-0435-460f-9d96-fde33e6d1c7c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:00.954869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e638-0435-460f-9d96-fde33e6d1c7c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:00.955993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e638-0435-460f-9d96-fde33e6d1c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:00.987253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-41e4e638-0435-460f-9d96-fde33e6d1c7c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:00.989527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-41e4e638-0435-460f-9d96-fde33e6d1c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:00.993560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a945684-9b0b-441a-babc-433d0522864a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:01.011734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:01.069062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:01.070271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:01.070752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:01.071221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:01.072606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:01.074257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:01.076986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:01.098544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 508/1016] 10.222.0.72 () {56 vars in 1112 bytes} [Thu Jun 16 14:43:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:01.101267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:01.133007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 509/1017] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:00 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:01.172701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e638-0435-460f-9d96-fde33e6d1c7c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:01.295473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a75231-0fc0-4f9c-a362-84ce04b02847 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:01.308982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:01.373166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:01.374092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:01.374377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:01.374648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:01.375811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:01.378341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:01.395433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e638-0435-460f-9d96-fde33e6d1c7c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:01.408144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 509/1018] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:43:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:43:01.496411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc992322-c818-43b2-9d1e-f95720c64742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:01.497871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc992322-c818-43b2-9d1e-f95720c64742 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:01.498531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc992322-c818-43b2-9d1e-f95720c64742 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:01.498878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc992322-c818-43b2-9d1e-f95720c64742 None admin] PATH_INFO: `/v3/services/image` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:01.500226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc992322-c818-43b2-9d1e-f95720c64742 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:01.503588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc992322-c818-43b2-9d1e-f95720c64742 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-cc992322-c818-43b2-9d1e-f95720c64742 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:01.527016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:01.530303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:43:01.533298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:01.537447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 510/1019] 10.222.0.72 () {56 vars in 1105 bytes} [Thu Jun 16 14:43:01 2022] GET /identity/v3/services/image => generated 86 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:01.539043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:01.597788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:01.712684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e299c-be3e-4de8-9b3e-2f62c190e699 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:01.713843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e299c-be3e-4de8-9b3e-2f62c190e699 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:01.714396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e299c-be3e-4de8-9b3e-2f62c190e699 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:01.717311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 511/1020] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:01.726672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eeb20-5618-40c8-8bec-f4acda6e3f10 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:01.727116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eeb20-5618-40c8-8bec-f4acda6e3f10 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:01.727573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eeb20-5618-40c8-8bec-f4acda6e3f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:01.765937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7def2ac-e275-4831-aa44-b47f017c9559 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:01.772433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-760eeb20-5618-40c8-8bec-f4acda6e3f10 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:01.774352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-760eeb20-5618-40c8-8bec-f4acda6e3f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:01.847741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:01.848745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:01.851671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:01.851671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:01.851671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] RBAC: Adding query filter params (name=image) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:01.852096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:01.853608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:01.864149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 510/1021] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:01.875109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:01.922331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 512/1022] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:01.962417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eeb20-5618-40c8-8bec-f4acda6e3f10 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:02.061918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:02.105233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:02.122814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869fd84-3974-4c8d-8f70-21ddf1f00091 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:02.143819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eeb20-5618-40c8-8bec-f4acda6e3f10 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:02.202945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:02.204166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:02.204679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:02.205755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:02.207130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] RBAC: Adding query filter params (type=image) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:02.207659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:02.210276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:02.215473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:02.217077 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:02.217670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:02.218175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:02.223006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:02.224044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 511/1023] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:02.225258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:02.236705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 513/1024] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:02.238132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:02.238589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:02.435405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2edf-e966-4a1d-8af7-5c71e03627eb None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:02.448545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2a376-17b9-4928-9f75-00323aa2a350 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:02.487379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:02.510658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36e7d-8ad1-47a7-addf-abece73e987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:02.510658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36e7d-8ad1-47a7-addf-abece73e987a None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:02.511248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36e7d-8ad1-47a7-addf-abece73e987a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:02.511682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36e7d-8ad1-47a7-addf-abece73e987a None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:02.513106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36e7d-8ad1-47a7-addf-abece73e987a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:02.513426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:02.514219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:02.514635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:02.515044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:02.515921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36e7d-8ad1-47a7-addf-abece73e987a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:02.516222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] RBAC: Adding query filter params (interface=public, service_id=584ad1bcc3b040d99504ea85505cd434, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:02.516600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:02.518837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:02.531960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 512/1025] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:43:02 2022] GET /identity/v3/endpoints?service_id=584ad1bcc3b040d99504ea85505cd434&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:02.535636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:02.561807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 514/1026] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:43:02 2022] POST /identity/v3/projects => generated 311 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:43:02.585336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:02.782393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5fa9f-c3a3-443f-a2ac-ce930c931452 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:02.865830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df729c-ccee-4ede-a343-d1c9e9f517ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:02.867062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df729c-ccee-4ede-a343-d1c9e9f517ff None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:02.867569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df729c-ccee-4ede-a343-d1c9e9f517ff None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:02.868032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df729c-ccee-4ede-a343-d1c9e9f517ff None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:02.869589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df729c-ccee-4ede-a343-d1c9e9f517ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:02.872347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df729c-ccee-4ede-a343-d1c9e9f517ff None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:02.886286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 513/1027] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:43:02 2022] GET /identity/v3/services => generated 1957 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:43:04.609078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a194f41-a4fc-44ac-8417-fafbf1bf1861 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:04.609914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a194f41-a4fc-44ac-8417-fafbf1bf1861 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:04.610058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a194f41-a4fc-44ac-8417-fafbf1bf1861 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:04.613027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 515/1028] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:04.620223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441fd1-bfe0-43f9-b785-48963e030dc7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:04.621231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441fd1-bfe0-43f9-b785-48963e030dc7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:04.621231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d441fd1-bfe0-43f9-b785-48963e030dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:04.641496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5d441fd1-bfe0-43f9-b785-48963e030dc7 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:04.643981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5d441fd1-bfe0-43f9-b785-48963e030dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:04.742206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 514/1029] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:04.766795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a194f41-a4fc-44ac-8417-fafbf1bf1861 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:05.000237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a194f41-a4fc-44ac-8417-fafbf1bf1861 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:05.071543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:05.073044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:05.073562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:05.074061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:05.095047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:05.112266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:05.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:43:05.114277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:05.115473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 516/1030] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Jun 16 14:43:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:05.121568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d441fd1-bfe0-43f9-b785-48963e030dc7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:05.144585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:05.274055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d441fd1-bfe0-43f9-b785-48963e030dc7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:05.319077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:05.319777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:05.320145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:05.320372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:05.321156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:05.321468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:05.322874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:05.339902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 515/1031] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Jun 16 14:43:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:05.341269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:05.403522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:05.538490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc41b0-22a1-4483-8811-662eb22e2640 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:05.609556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79622d3-ef68-455c-8e2d-06f2945d6aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:05.610963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79622d3-ef68-455c-8e2d-06f2945d6aad None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:05.612025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79622d3-ef68-455c-8e2d-06f2945d6aad None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:05.612527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79622d3-ef68-455c-8e2d-06f2945d6aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:05.635777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79622d3-ef68-455c-8e2d-06f2945d6aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:05.638892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79622d3-ef68-455c-8e2d-06f2945d6aad None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-b79622d3-ef68-455c-8e2d-06f2945d6aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:05.669066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:05.670709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:05.672421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:43:05.672421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:05.672421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:05.672421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:05.672421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 517/1032] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:05.677829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:05.730560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34fdc9-7088-4420-95ff-147c1a7d50f3 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:05.731145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34fdc9-7088-4420-95ff-147c1a7d50f3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:05.731617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34fdc9-7088-4420-95ff-147c1a7d50f3 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:05.733399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 518/1033] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:05.740482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737012cb-155f-4299-8047-0188039c7d91 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:05.741047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737012cb-155f-4299-8047-0188039c7d91 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:05.741538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737012cb-155f-4299-8047-0188039c7d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:05.775380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-737012cb-155f-4299-8047-0188039c7d91 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:05.776608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-737012cb-155f-4299-8047-0188039c7d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:05.787787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:05.835291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced9cb5-6c57-42fc-be44-439e6a523d33 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:05.899608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 519/1034] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:05.903824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:05.904622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:05.905059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:05.905466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:05.906555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:05.906913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:05.909399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:05.930692 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 516/1035] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:05.933491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737012cb-155f-4299-8047-0188039c7d91 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:05.937371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:05.966191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:06.115780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52c78a-a955-4880-a66a-50f96fd3e541 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:06.117468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737012cb-155f-4299-8047-0188039c7d91 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:06.181999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6951496-1e3e-4d17-952c-c394abf15b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:06.183206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6951496-1e3e-4d17-952c-c394abf15b55 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.183668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6951496-1e3e-4d17-952c-c394abf15b55 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.183860 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa4c4a-fa8e-40f6-8bad-38a30998ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:06.184127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6951496-1e3e-4d17-952c-c394abf15b55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:06.184534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa4c4a-fa8e-40f6-8bad-38a30998ee3c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.184821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa4c4a-fa8e-40f6-8bad-38a30998ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.185137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa4c4a-fa8e-40f6-8bad-38a30998ee3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:06.186500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa4c4a-fa8e-40f6-8bad-38a30998ee3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:06.188006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa4c4a-fa8e-40f6-8bad-38a30998ee3c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:06.190627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:06.190879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 520/1036] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:06.192365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4bc31-7b28-4213-9a50-32bb6b438b20 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.192646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4bc31-7b28-4213-9a50-32bb6b438b20 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.192963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4bc31-7b28-4213-9a50-32bb6b438b20 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:06.194412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 521/1037] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:06.199663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4bc31-7b28-4213-9a50-32bb6b438b20 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:06.204007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6951496-1e3e-4d17-952c-c394abf15b55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:06.206533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6951496-1e3e-4d17-952c-c394abf15b55 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b6951496-1e3e-4d17-952c-c394abf15b55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:43:06.224187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:06.226318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:06.228179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 517/1038] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:43:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:06.229551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bb4b1-606f-4698-a297-ff25a34cc463 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.229551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bb4b1-606f-4698-a297-ff25a34cc463 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.229798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bb4b1-606f-4698-a297-ff25a34cc463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:06.249794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2b4bb4b1-606f-4698-a297-ff25a34cc463 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:06.251277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2b4bb4b1-606f-4698-a297-ff25a34cc463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:06.348775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:06.366221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 518/1039] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:06.368205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bb4b1-606f-4698-a297-ff25a34cc463 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:06.373732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4bc31-7b28-4213-9a50-32bb6b438b20 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:06.448341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:06.449190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.449651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.450104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:06.451366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:06.453657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:06.488362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 522/1040] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:43:06 2022] POST /identity/v3/users => generated 285 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:43:06.496106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:06.579741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bb4b1-606f-4698-a297-ff25a34cc463 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:06.631229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:06.648617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:06.649437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.649785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.650129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:06.651415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:06.651800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:06.653447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:06.679488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 519/1041] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:43:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:06.684305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217fce5-54e9-4460-a97b-c651e3162b58 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:06.686012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:06.743254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:06.744312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.744732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.745207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] PATH_INFO: `/v3/services/image` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:06.746440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:06.748153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:06.764187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:06.765833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:06.768096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 523/1042] 10.222.0.72 () {56 vars in 1105 bytes} [Thu Jun 16 14:43:06 2022] GET /identity/v3/services/image => generated 86 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:06.773064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:06.787162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:06.904037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63be71-2001-4348-a9f3-332c963710b1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:06.947864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f15bd-9737-452f-adad-1463952663dd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:06.977270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:06.978028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:06.978311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:06.978583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:07.007853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:07.008693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:07.009043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:07.009325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:07.010778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] RBAC: Adding query filter params (name=image) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:07.011151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:07.012705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:07.019521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 524/1043] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:07.025217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:07.076242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:07.097759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:07.099266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:07.139539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:07.187034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5794209-54ed-4de1-90d8-8c0862b558f1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:07.238546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:07.239231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:07.239516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:07.240585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:07.241369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] RBAC: Adding query filter params (type=image) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:07.241684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:07.243502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:07.258355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 525/1044] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:07.260697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:07.261639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 520/1045] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:06 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:43:07.270507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:07.422080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:07.427678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287ee59-46f6-4cdb-a90f-11ed09dc37e2 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:07.462370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebf042-6a7d-4a5a-b9ee-fa110d29a8dd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:07.487245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:07.488248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:07.488541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:07.488827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:07.489651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:07.491272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:07.495675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:07.528999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 526/1046] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:43:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:43:07.560699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:07.562170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:07.562659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:07.563153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:07.585654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:07.588136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:07.626558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:07.628125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:07.629093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:43:07.629093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:07.629093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:07.629093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:07.634927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 521/1047] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:07.636440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:07.826821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ff38c-521d-4e0b-ac79-e2a3b169f6c4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:07.900832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:07.901980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:07.902456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:07.902920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:07.904121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:07.904640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:07.906957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:07.925982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 527/1048] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:07.932294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:08.120341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035a315-7a55-4740-8e44-89f01e86ab05 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:08.163896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:08.174237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:08.175079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:08.175370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:08.175648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:08.184563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:08.186352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:43:08.203120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:08.205786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:08.207868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 522/1049] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:08.213256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:08.320258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:08.432563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e447e-df53-413b-939e-e0e42289e23c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:08.496610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1574b6-b2d7-46da-9b38-685f7a2c8ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:08.498034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1574b6-b2d7-46da-9b38-685f7a2c8ef8 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:08.498510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1574b6-b2d7-46da-9b38-685f7a2c8ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:08.499016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1574b6-b2d7-46da-9b38-685f7a2c8ef8 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:08.500644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1574b6-b2d7-46da-9b38-685f7a2c8ef8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:08.501465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1574b6-b2d7-46da-9b38-685f7a2c8ef8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:08.504951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1574b6-b2d7-46da-9b38-685f7a2c8ef8 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:08.536882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 528/1050] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:08.540377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:08.755358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e8cd9-e28f-4991-91a3-3d22131fefe5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:08.829296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad90f6-e9a9-4ad1-ac2b-62c032e21803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:08.831451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad90f6-e9a9-4ad1-ac2b-62c032e21803 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:08.832223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad90f6-e9a9-4ad1-ac2b-62c032e21803 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:08.832990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad90f6-e9a9-4ad1-ac2b-62c032e21803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:08.835042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad90f6-e9a9-4ad1-ac2b-62c032e21803 None admin] RBAC: Adding query filter params (role.id=3aafc58b4d6743218143e36035ba7b4f, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0b8661ae3402445c9c027e739983f26e) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:08.836155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad90f6-e9a9-4ad1-ac2b-62c032e21803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:08.839559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad90f6-e9a9-4ad1-ac2b-62c032e21803 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:08.852525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 523/1051] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:08 2022] GET /identity/v3/role_assignments?user.id=0b8661ae3402445c9c027e739983f26e&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=3aafc58b4d6743218143e36035ba7b4f => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:09.967138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ad33d-606b-4a75-a4e3-e913e3d72c46 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:09.968000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ad33d-606b-4a75-a4e3-e913e3d72c46 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:09.968154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ad33d-606b-4a75-a4e3-e913e3d72c46 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:09.972908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 529/1052] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:09.980077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da7bf-e7dc-426e-8238-0052e0781898 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:09.981118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da7bf-e7dc-426e-8238-0052e0781898 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:09.981118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da7bf-e7dc-426e-8238-0052e0781898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:10.008437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-258da7bf-e7dc-426e-8238-0052e0781898 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:10.010481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-258da7bf-e7dc-426e-8238-0052e0781898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:10.105084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:10.143771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 524/1053] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:10.184951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ad33d-606b-4a75-a4e3-e913e3d72c46 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:10.326312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ad33d-606b-4a75-a4e3-e913e3d72c46 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:10.347361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:10.360928 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4310c-194a-40fa-a0af-476d5a640a33 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:10.361947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4310c-194a-40fa-a0af-476d5a640a33 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:10.362383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4310c-194a-40fa-a0af-476d5a640a33 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:10.365109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 525/1054] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:10.374438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959b4e8-fe86-4358-9c1a-4532e7a135d3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:10.374913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959b4e8-fe86-4358-9c1a-4532e7a135d3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:10.375382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959b4e8-fe86-4358-9c1a-4532e7a135d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:10.383773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:10.384597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:10.384918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:10.385202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:10.399211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7959b4e8-fe86-4358-9c1a-4532e7a135d3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:10.400329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7959b4e8-fe86-4358-9c1a-4532e7a135d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:10.400662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:10.411170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:10.413798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:10.415778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 530/1055] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:43:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:10.418772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:10.511904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 526/1056] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:10.551156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959b4e8-fe86-4358-9c1a-4532e7a135d3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:10.589535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c220111-0f1e-4385-b553-33490ec82d7d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:10.651446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:10.653297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:10.654056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:10.654978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:10.656913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:10.657477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:10.660272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:10.677835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 531/1057] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:43:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:10.684092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:10.713417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959b4e8-fe86-4358-9c1a-4532e7a135d3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:10.744000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:10.745426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:10.776966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:10.778181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:10.778572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:10.778912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:10.858006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ebf9e-0835-4e34-b8fe-d97d5c138e40 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:10.900975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:10.904034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:10.910788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:10.912043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:10.912559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:10.913089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] PATH_INFO: `/v3/users/dd2c85ba9e8b4bd6adec293c6af39532` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:10.931455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] RBAC: Authorizing `identity:get_user(user_id=dd2c85ba9e8b4bd6adec293c6af39532)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:10.933498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:10.935572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 532/1058] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:10 2022] GET /identity/v3/users/dd2c85ba9e8b4bd6adec293c6af39532 => generated 285 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:10.944521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:11.086172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 527/1059] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:43:11.092226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:11.109450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:11.114540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:11.127439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a99acc-435c-4525-9dde-ea9ef8a057ef None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:11.176762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45de0b-6c58-4bcd-8537-c50e76abae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:11.177392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45de0b-6c58-4bcd-8537-c50e76abae7e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:11.177687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45de0b-6c58-4bcd-8537-c50e76abae7e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:11.177947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45de0b-6c58-4bcd-8537-c50e76abae7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:11.266872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce974d-9736-4b02-ae03-c816466b4029 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:11.294057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45de0b-6c58-4bcd-8537-c50e76abae7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:11.297056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45de0b-6c58-4bcd-8537-c50e76abae7e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:11.321721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:11.322534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:11.322826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:11.323069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:11.335903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:11.337832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:11.358729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:11.360387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:11.362564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:43:11.362564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:11.362564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:11.362564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:11.366035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 528/1060] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:11.366858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:11.459464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:11.466027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:11.475774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 533/1061] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:43:11.482257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ac2e9-efda-40fd-ae5e-8bc3ef7b32fd None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:11.482536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ac2e9-efda-40fd-ae5e-8bc3ef7b32fd None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:11.482803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ac2e9-efda-40fd-ae5e-8bc3ef7b32fd None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:11.484517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 534/1062] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:11.485172 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ac2e9-efda-40fd-ae5e-8bc3ef7b32fd None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:11.518290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee2a7a-2a06-46a9-b4e8-c68684c5b682 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:11.564983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e0b9e-6014-42ca-a19c-36552056c1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:11.565756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e0b9e-6014-42ca-a19c-36552056c1f5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:11.566118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e0b9e-6014-42ca-a19c-36552056c1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:11.566539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e0b9e-6014-42ca-a19c-36552056c1f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:11.567874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e0b9e-6014-42ca-a19c-36552056c1f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:11.568414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e0b9e-6014-42ca-a19c-36552056c1f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:11.570909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e0b9e-6014-42ca-a19c-36552056c1f5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:11.588912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 529/1063] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:11.591160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a5b0-9777-4868-b980-bf1fd6bab2dd None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:11.591658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a5b0-9777-4868-b980-bf1fd6bab2dd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:11.592158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a5b0-9777-4868-b980-bf1fd6bab2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:11.619899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-af71a5b0-9777-4868-b980-bf1fd6bab2dd None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:11.621986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-af71a5b0-9777-4868-b980-bf1fd6bab2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:11.649442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ac2e9-efda-40fd-ae5e-8bc3ef7b32fd None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:11.700912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:11.701842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:11.702124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:11.702410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:11.703781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] RBAC: Authorizing `identity:get_project(project_id=47aa8e95728c4c20aed763feb0b5a209)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:11.705457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:11.707547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 535/1064] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:11 2022] GET /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209 => generated 311 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:11.718879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:11.746543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 530/1065] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:11.772056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71a5b0-9777-4868-b980-bf1fd6bab2dd None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:11.820567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:11.872299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:11.904944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953c0e8-01c9-4c4c-867f-82098c7d83b6 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:11.950225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71a5b0-9777-4868-b980-bf1fd6bab2dd None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:11.966302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:11.967598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:11.968964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:11.969161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:11.970655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=47aa8e95728c4c20aed763feb0b5a209, user.id=dd2c85ba9e8b4bd6adec293c6af39532) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:11.971249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:11.973336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:11.985609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 536/1066] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:11 2022] GET /identity/v3/role_assignments?user.id=dd2c85ba9e8b4bd6adec293c6af39532&scope.project.id=47aa8e95728c4c20aed763feb0b5a209&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:12.015883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:12.017445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:12.017978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:12.018446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:12.038229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:12.047318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:12.050159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:43:12.052238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:12.054841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 531/1067] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Jun 16 14:43:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:12.056595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:12.210615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1520-c1a7-4ec4-a0f6-3863d6f408e1 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:12.253157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:12.272906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:12.273922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:12.274209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:12.274492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:12.275273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:12.275583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:12.277107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:12.289188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 537/1068] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Jun 16 14:43:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:12.295823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:12.444650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248fa8b-fcf4-4bac-b9cc-bbbf37550266 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:12.481400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:12.518047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:12.519657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:12.520174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:12.520688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:12.542151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:12.545079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:12.582287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:12.583468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:12.584717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:43:12.584717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:12.584717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:12.584717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:12.584717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 532/1069] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:12.591419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:12.747872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242ed27-99eb-41e2-aacc-6b81665f911a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:12.803199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:12.804007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:12.804376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:12.804744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:12.805651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:12.806054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:12.807556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:12.823858 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 538/1070] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:12.830074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:12.992346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c48630-f896-4343-bb05-b18e6459857f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:13.036548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:13.037578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:13.037988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:13.038381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:13.048266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:13.051216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:43:13.070638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:13.072656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:13.074773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 533/1071] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:43:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:13.082443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:13.160909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:13.261835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ba9ec-29f9-4e32-aa70-a6c0ec6d3b08 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:13.331142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:13.332171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:13.332673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:13.333207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:13.334768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:13.335344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:13.337971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:13.373969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 539/1072] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:43:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:13.383637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:13.444740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:13.571335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d59e2-75c6-4e4f-9113-2ecae223c074 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:13.638768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:13.640068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:13.640582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:13.641127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:13.662980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:13.665953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:13.701438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:13.702542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:13.703468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:43:13.703468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:13.703468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:13.703468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:13.703695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 534/1073] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:13.713202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:13.880558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e837f-9e70-4a4d-8605-363b40f9984a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:13.883645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:13.948241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:13.949211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:13.949703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:13.950183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:13.951509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:13.952055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:13.954111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:13.969907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 540/1074] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:13.976149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:14.105601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:14.138274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc1f8b-3b42-4edc-b9ee-1ddb5a2a8566 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:14.184813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bce504-f275-4008-a8b6-2a8eadcb0664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:14.185670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce504-f275-4008-a8b6-2a8eadcb0664 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:14.185955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce504-f275-4008-a8b6-2a8eadcb0664 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:14.186285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bce504-f275-4008-a8b6-2a8eadcb0664 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:14.196203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce504-f275-4008-a8b6-2a8eadcb0664 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:14.198673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bce504-f275-4008-a8b6-2a8eadcb0664 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d4bce504-f275-4008-a8b6-2a8eadcb0664 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:43:14.214872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:14.216702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:14.218507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 535/1075] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:14.223988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:14.353470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba2f27-8419-4dfc-ac70-609a0b68d909 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:14.353922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba2f27-8419-4dfc-ac70-609a0b68d909 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:14.354368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba2f27-8419-4dfc-ac70-609a0b68d909 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:14.356018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 536/1076] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:14.365247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21ee28-870f-499e-979f-c957fdf823a5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:14.365629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21ee28-870f-499e-979f-c957fdf823a5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:14.366092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21ee28-870f-499e-979f-c957fdf823a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:14.392971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8f21ee28-870f-499e-979f-c957fdf823a5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:14.395166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8f21ee28-870f-499e-979f-c957fdf823a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:14.403440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000ebc8-567c-4462-88e8-b61bd72e91ec None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:14.453886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:14.454471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:14.454808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:14.455081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:14.455907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:14.456227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:14.457757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:14.477967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 541/1077] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:14.484786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:14.509358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 537/1078] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:14.520684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:14.539961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21ee28-870f-499e-979f-c957fdf823a5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:14.633137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:14.646970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c572412-ded5-4e83-baf9-a4a03bcb0d38 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:14.678676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21ee28-870f-499e-979f-c957fdf823a5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:14.696164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:14.697345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:14.697549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:14.697909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0b8661ae3402445c9c027e739983f26e/roles/3aafc58b4d6743218143e36035ba7b4f` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:14.701202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=0b8661ae3402445c9c027e739983f26e, role_id=3aafc58b4d6743218143e36035ba7b4f)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:14.702983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:14.726705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 542/1079] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:43:14 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/0b8661ae3402445c9c027e739983f26e/roles/3aafc58b4d6743218143e36035ba7b4f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:43:14.728672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:14.729875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:14.730237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:14.730614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:14.747217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:14.755780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:14.757643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:14.759276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 538/1080] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:14.764554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:14.917871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:14.952435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad6ab-96e9-4c72-a008-3fdc71a86efb None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:15.016297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:15.017315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:15.017702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:15.018043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:15.019617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:15.020158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:15.022739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:15.038956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 543/1081] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:43:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:15.049378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:15.203883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083918d-c6ed-4eb6-b105-ca212c48d889 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:15.258690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:15.265922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:15.267067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:15.267414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:15.267714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] PATH_INFO: `/v3/users/dd2c85ba9e8b4bd6adec293c6af39532` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:15.269568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] RBAC: Authorizing `identity:get_user(user_id=dd2c85ba9e8b4bd6adec293c6af39532)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:15.271037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:15.272778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 539/1082] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:15 2022] GET /identity/v3/users/dd2c85ba9e8b4bd6adec293c6af39532 => generated 285 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:15.279565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:15.430716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a392da-8f4f-4fbc-a927-f24d7ff3682e None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:15.473643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4d7c7-3a51-478c-9a8a-2e5e929d7d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:15.474510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4d7c7-3a51-478c-9a8a-2e5e929d7d94 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:15.474790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4d7c7-3a51-478c-9a8a-2e5e929d7d94 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:15.475066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4d7c7-3a51-478c-9a8a-2e5e929d7d94 None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:15.476930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4d7c7-3a51-478c-9a8a-2e5e929d7d94 None admin] RBAC: Authorizing `identity:get_project(project_id=47aa8e95728c4c20aed763feb0b5a209)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:15.479450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4d7c7-3a51-478c-9a8a-2e5e929d7d94 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:15.487768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 544/1083] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:15 2022] GET /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209 => generated 311 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:15.489002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:15.617938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8427ee-fe62-4b8d-81d0-1568671544c1 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:15.688885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a144d-394b-4edc-b18c-691094e0eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:15.690921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a144d-394b-4edc-b18c-691094e0eca8 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:15.691433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a144d-394b-4edc-b18c-691094e0eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:15.691933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a144d-394b-4edc-b18c-691094e0eca8 None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209/users/dd2c85ba9e8b4bd6adec293c6af39532/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:15.695948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a144d-394b-4edc-b18c-691094e0eca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=47aa8e95728c4c20aed763feb0b5a209, user_id=dd2c85ba9e8b4bd6adec293c6af39532, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:15.698897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a144d-394b-4edc-b18c-691094e0eca8 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:15.726783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 540/1084] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:43:15 2022] PUT /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209/users/dd2c85ba9e8b4bd6adec293c6af39532/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:43:17.158510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f25e0-3cd1-41b3-a656-965f0233feca None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:17.159314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f25e0-3cd1-41b3-a656-965f0233feca None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:17.159314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f25e0-3cd1-41b3-a656-965f0233feca None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:17.160445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 545/1085] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:17.170359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d30329-8a06-462d-93ab-8707b5eb8f48 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:17.170953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d30329-8a06-462d-93ab-8707b5eb8f48 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:17.171359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d30329-8a06-462d-93ab-8707b5eb8f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:17.203583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a9d30329-8a06-462d-93ab-8707b5eb8f48 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:17.205829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a9d30329-8a06-462d-93ab-8707b5eb8f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:17.306292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:17.320970 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 541/1086] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:17.348033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f25e0-3cd1-41b3-a656-965f0233feca None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:17.374135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:17.513666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f25e0-3cd1-41b3-a656-965f0233feca None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:17.560990 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:17.561838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:17.562128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:17.562407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:17.576041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:17.585204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:17.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 14:43:17.589358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:17.591751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 546/1087] 10.222.0.72 () {56 vars in 1120 bytes} [Thu Jun 16 14:43:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:17.592763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d30329-8a06-462d-93ab-8707b5eb8f48 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:17.748446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d30329-8a06-462d-93ab-8707b5eb8f48 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:17.794392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:17.795155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:17.795448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:17.795729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:17.796518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:17.796831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:17.798317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:17.809735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 542/1088] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Jun 16 14:43:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:17.816884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:17.932607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:17.998330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08584b-eca0-464b-b92b-0980841d8f10 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:18.052542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b9170-e4ad-4c32-bde9-37e43f6942a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:18.053392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b9170-e4ad-4c32-bde9-37e43f6942a7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.053766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b9170-e4ad-4c32-bde9-37e43f6942a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.054169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b9170-e4ad-4c32-bde9-37e43f6942a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.068464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b9170-e4ad-4c32-bde9-37e43f6942a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:18.070327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b9170-e4ad-4c32-bde9-37e43f6942a7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-b86b9170-e4ad-4c32-bde9-37e43f6942a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:18.090664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:18.092691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:18.093996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:43:18.093996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:18.093996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:18.093996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:18.093996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 547/1089] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:18.104508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:18.179261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:18.183059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a856d-984b-401e-b34e-4b405e55a4a5 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.184024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a856d-984b-401e-b34e-4b405e55a4a5 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.184174 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a856d-984b-401e-b34e-4b405e55a4a5 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.185870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 548/1090] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:18.193470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35706-106a-40f7-90c4-2ecba2027874 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.193906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35706-106a-40f7-90c4-2ecba2027874 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.194324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35706-106a-40f7-90c4-2ecba2027874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.212965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-27c35706-106a-40f7-90c4-2ecba2027874 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:18.214212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-27c35706-106a-40f7-90c4-2ecba2027874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:18.256402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc66166-196a-4445-ae1b-bf4956dc46e6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:18.307839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 549/1091] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:18.319829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:18.321050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.321562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.322052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.323378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:18.323861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:18.326535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:18.338607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35706-106a-40f7-90c4-2ecba2027874 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:18.351544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 543/1092] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:18.360362 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:18.418977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:18.485281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35706-106a-40f7-90c4-2ecba2027874 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:18.516708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833e8ec-6a16-4e64-80e1-844d28bdbdf5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:18.531328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:18.532146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.532429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.532782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.537399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:18.546462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:18.557601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:18.559698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:18.561464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 550/1093] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:18.567766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:18.577357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:18.578718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.579200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.586173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.600783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:18.603642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:43:18.633887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:18.635911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:18.637810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 544/1094] 10.222.0.72 () {56 vars in 1093 bytes} [Thu Jun 16 14:43:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:18.645677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:18.721839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df32b1-2c68-4d32-a76c-756d45d044d0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:18.741001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:18.770163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:18.770927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.771330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.771673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.773024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:18.773509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:18.775944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:18.786339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 551/1095] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:43:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:18.795268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:18.802371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b92401-3664-4baf-9ee4-b3d3b9864e4a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:18.850013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:18.850971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:18.851418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:18.851756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:18.852743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:18.853203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:18.855289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:18.879872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 545/1096] 10.222.0.72 () {56 vars in 1133 bytes} [Thu Jun 16 14:43:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:18.888695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:18.932345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:18.957987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa9395-0276-42e6-9200-cbe30ea85af5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:19.007407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:19.008875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:19.009277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:19.009691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] PATH_INFO: `/v3/users/dd2c85ba9e8b4bd6adec293c6af39532` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:19.011719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=dd2c85ba9e8b4bd6adec293c6af39532)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:19.014422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:19.017962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 552/1097] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:18 2022] GET /identity/v3/users/dd2c85ba9e8b4bd6adec293c6af39532 => generated 285 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:19.027274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:19.052134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57190587-1d1b-45d9-91e9-d86e55b26e72 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:19.097705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:19.102871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:19.103764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:19.104239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:19.104711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:19.170825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72ab6e-317d-445d-a091-24a15394a2d4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:19.193031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:19.195041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:19.212534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:19.213323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:19.213520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:19.213792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:19.236913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:19.307320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:19.308890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:19.335667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 546/1098] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:43:19.342209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:19.367962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:19.441958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 553/1099] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:43:19.448481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:19.486742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb925835-0725-44a1-9f4b-f6a8d2e449df None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:19.524029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:19.531013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:19.531828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:19.532111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:19.532384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:19.549995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:19.552682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:19.582562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:19.583749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:19.585263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:43:19.585263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:19.585263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:43:19.585263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:19.585263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 547/1100] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:19.593892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:19.602117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cef59-6d8f-430c-968d-8b6f353a4f3d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:19.657427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:19.659160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:19.659635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:19.660212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:19.662410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] RBAC: Authorizing `identity:get_project(project_id=47aa8e95728c4c20aed763feb0b5a209)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:19.664880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:19.667921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 554/1101] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:19 2022] GET /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209 => generated 311 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:19.677249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:19.706173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:19.763697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76e9cc-3184-41ab-8a0a-b81c0564c3f6 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:19.807994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:19.808699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:19.808978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:19.809261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:19.810061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:19.810342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:19.812268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:19.825954 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 548/1102] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:19.832794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:19.850585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34f178-02d0-4ccd-983f-aa25bf2abb6c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:19.897302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:19.912009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:19.912600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:19.912925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:19.913215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:19.914093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=47aa8e95728c4c20aed763feb0b5a209, user.id=dd2c85ba9e8b4bd6adec293c6af39532) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:19.914413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:19.916160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:19.926002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 555/1103] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:19 2022] GET /identity/v3/role_assignments?user.id=dd2c85ba9e8b4bd6adec293c6af39532&scope.project.id=47aa8e95728c4c20aed763feb0b5a209&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:19.997926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4933fdb-c75a-4b73-8c7d-aa6908bb6538 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:20.048805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:20.050203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:20.050854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:20.051259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:20.067449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:20.069113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:43:20.088503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:20.090205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:20.091212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 549/1104] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:20.096496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:20.212582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:20.249484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc3a4a-f102-4041-930d-3922b713fe4c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:20.294722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f953-ebbe-460b-9048-7ebe19110036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:20.295495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f953-ebbe-460b-9048-7ebe19110036 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:20.295851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f953-ebbe-460b-9048-7ebe19110036 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:20.296197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f953-ebbe-460b-9048-7ebe19110036 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:20.297147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4f953-ebbe-460b-9048-7ebe19110036 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:20.297535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4f953-ebbe-460b-9048-7ebe19110036 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:20.299173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4f953-ebbe-460b-9048-7ebe19110036 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:20.313999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 556/1105] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:20.323886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:20.460597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cab549-567f-4514-adaf-06e48bdf1405 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:20.462687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:20.514789 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bed846-3460-41c5-8b68-a5e25802c321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:20.515476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed846-3460-41c5-8b68-a5e25802c321 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:20.515756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed846-3460-41c5-8b68-a5e25802c321 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:20.516031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bed846-3460-41c5-8b68-a5e25802c321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:20.516907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed846-3460-41c5-8b68-a5e25802c321 None admin] RBAC: Adding query filter params (role.id=3aafc58b4d6743218143e36035ba7b4f, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=0b8661ae3402445c9c027e739983f26e) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:20.517301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed846-3460-41c5-8b68-a5e25802c321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:20.519035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bed846-3460-41c5-8b68-a5e25802c321 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:20.526445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 550/1106] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:20 2022] GET /identity/v3/role_assignments?user.id=0b8661ae3402445c9c027e739983f26e&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=3aafc58b4d6743218143e36035ba7b4f => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:22.464871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b62a35-57a6-4642-a821-59a72b4c4090 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:22.465561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b62a35-57a6-4642-a821-59a72b4c4090 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:22.465561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b62a35-57a6-4642-a821-59a72b4c4090 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:22.466895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 557/1107] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:22.473140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d276137-fc96-4a60-90a2-93e1866e7fb7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:22.473770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d276137-fc96-4a60-90a2-93e1866e7fb7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:22.473770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d276137-fc96-4a60-90a2-93e1866e7fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:22.493495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9d276137-fc96-4a60-90a2-93e1866e7fb7 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:22.494744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9d276137-fc96-4a60-90a2-93e1866e7fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:22.580987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 551/1108] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:22.608144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b62a35-57a6-4642-a821-59a72b4c4090 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:22.794199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b62a35-57a6-4642-a821-59a72b4c4090 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:22.850141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a040f9-d866-4c97-94ad-691dfb31b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:22.851438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a040f9-d866-4c97-94ad-691dfb31b1db None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:22.851925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a040f9-d866-4c97-94ad-691dfb31b1db None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:22.852398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a040f9-d866-4c97-94ad-691dfb31b1db None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:22.854547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a040f9-d866-4c97-94ad-691dfb31b1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:22.857178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a040f9-d866-4c97-94ad-691dfb31b1db None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:22.860358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 558/1109] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:22.867416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d276137-fc96-4a60-90a2-93e1866e7fb7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:22.982855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:22.991062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d276137-fc96-4a60-90a2-93e1866e7fb7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:23.033222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a247b-f571-4e5f-9a5d-ae20ce409007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:23.034426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a247b-f571-4e5f-9a5d-ae20ce409007 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:23.034931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a247b-f571-4e5f-9a5d-ae20ce409007 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:23.035357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a247b-f571-4e5f-9a5d-ae20ce409007 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:23.036131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a247b-f571-4e5f-9a5d-ae20ce409007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:23.037671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a247b-f571-4e5f-9a5d-ae20ce409007 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:23.063945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 552/1110] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:43:22 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:43:25.627576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eefa00-d00f-4f49-a39d-a7a236e6de0e None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:25.628216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eefa00-d00f-4f49-a39d-a7a236e6de0e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:25.628216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eefa00-d00f-4f49-a39d-a7a236e6de0e None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:25.629879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 559/1111] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:25.636252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1138-bba6-4a3e-828a-7bf1a43c6932 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:25.636891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1138-bba6-4a3e-828a-7bf1a43c6932 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:25.636891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1138-bba6-4a3e-828a-7bf1a43c6932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:25.657439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5b3d1138-bba6-4a3e-828a-7bf1a43c6932 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:25.659213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5b3d1138-bba6-4a3e-828a-7bf1a43c6932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:25.749793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 553/1112] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:25.781724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eefa00-d00f-4f49-a39d-a7a236e6de0e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:25.787884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:25.960534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eefa00-d00f-4f49-a39d-a7a236e6de0e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:26.002623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:26.003579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:26.004043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:26.004247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] PATH_INFO: `/v3/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:26.005969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] RBAC: Authorizing `identity:get_role(role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:26.007531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:26.009040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 560/1113] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:25 2022] GET /identity/v3/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 223 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:26.015535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d1138-bba6-4a3e-828a-7bf1a43c6932 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:26.120995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:26.145639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d1138-bba6-4a3e-828a-7bf1a43c6932 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:26.190346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:26.191451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:26.191839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:26.192207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] PATH_INFO: `/v3/users/2731105fbde540beb2907be40540fec5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:26.209247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] RBAC: Authorizing `identity:get_user(user_id=2731105fbde540beb2907be40540fec5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:26.210786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:26.212655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 554/1114] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:26 2022] GET /identity/v3/users/2731105fbde540beb2907be40540fec5 => generated 286 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:26.218907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:26.310432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:26.349667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c876c6-6719-4bf0-817c-fe51706357d2 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:26.391749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:26.392495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:26.392824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:26.393199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:26.502256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:26.504999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:26.603457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:26.644235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 561/1115] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:43:26.651304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:26.796638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce638a4c-e6a0-4bf1-beb8-300bbac9b73b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:26.847896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93099be8-299b-483f-9619-f804cd4d18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:26.848973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93099be8-299b-483f-9619-f804cd4d18ed None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:26.849343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93099be8-299b-483f-9619-f804cd4d18ed None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:26.849616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93099be8-299b-483f-9619-f804cd4d18ed None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:26.851603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93099be8-299b-483f-9619-f804cd4d18ed None admin] RBAC: Authorizing `identity:get_project(project_id=47aa8e95728c4c20aed763feb0b5a209)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:26.853273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93099be8-299b-483f-9619-f804cd4d18ed None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:26.855845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 555/1116] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:26 2022] GET /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209 => generated 311 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:26.862911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:27.000625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1675d-be5b-4efa-b839-0a20cc1f4a70 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:27.047597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded40d0-2809-47c1-bd18-8a8e6425387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:27.048267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded40d0-2809-47c1-bd18-8a8e6425387f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:27.048556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded40d0-2809-47c1-bd18-8a8e6425387f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:27.048954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded40d0-2809-47c1-bd18-8a8e6425387f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:27.050168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded40d0-2809-47c1-bd18-8a8e6425387f None admin] RBAC: Adding query filter params (role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=47aa8e95728c4c20aed763feb0b5a209, user.id=2731105fbde540beb2907be40540fec5) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:27.050442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded40d0-2809-47c1-bd18-8a8e6425387f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:27.052116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded40d0-2809-47c1-bd18-8a8e6425387f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:27.060503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 562/1117] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:26 2022] GET /identity/v3/role_assignments?user.id=2731105fbde540beb2907be40540fec5&scope.project.id=47aa8e95728c4c20aed763feb0b5a209&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:29.379504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fbe65-cf19-4023-a1b0-072ee46dab28 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:29.380164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fbe65-cf19-4023-a1b0-072ee46dab28 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:29.380164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fbe65-cf19-4023-a1b0-072ee46dab28 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:29.381483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 556/1118] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:29.387779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfb05d-5f55-4b5b-b886-854efd102a8e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:29.388638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfb05d-5f55-4b5b-b886-854efd102a8e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:29.388797 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfb05d-5f55-4b5b-b886-854efd102a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:29.410612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-47bfb05d-5f55-4b5b-b886-854efd102a8e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:29.411855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-47bfb05d-5f55-4b5b-b886-854efd102a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:29.448859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:29.500297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 563/1119] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:29.523743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fbe65-cf19-4023-a1b0-072ee46dab28 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:29.536627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:29.679741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fbe65-cf19-4023-a1b0-072ee46dab28 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:29.733789 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:29.735106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:29.735522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:29.735932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] PATH_INFO: `/v3/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:29.738693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] RBAC: Authorizing `identity:get_role(role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:29.741238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:29.743834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 557/1120] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:29 2022] GET /identity/v3/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 223 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:29.750151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfb05d-5f55-4b5b-b886-854efd102a8e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:29.892764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfb05d-5f55-4b5b-b886-854efd102a8e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:29.960898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:29.962431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:29.962431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:29.962431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] PATH_INFO: `/v3/users/2731105fbde540beb2907be40540fec5` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:29.964052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] RBAC: Authorizing `identity:get_user(user_id=2731105fbde540beb2907be40540fec5)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:29.965576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:29.967376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 564/1121] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:29 2022] GET /identity/v3/users/2731105fbde540beb2907be40540fec5 => generated 286 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:29.974375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:30.106266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:30.150320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d7e39-fb09-4c91-a2e7-9ebf0a4b9a1b None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:30.211713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092851e-66c1-4c0a-a22d-30a38ca8841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:30.222947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092851e-66c1-4c0a-a22d-30a38ca8841d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:30.223500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092851e-66c1-4c0a-a22d-30a38ca8841d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:30.223854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092851e-66c1-4c0a-a22d-30a38ca8841d None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:30.225984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092851e-66c1-4c0a-a22d-30a38ca8841d None admin] RBAC: Authorizing `identity:get_project(project_id=47aa8e95728c4c20aed763feb0b5a209)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:30.227847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092851e-66c1-4c0a-a22d-30a38ca8841d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:30.229986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 558/1122] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:29 2022] GET /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209 => generated 311 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:30.240886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:30.285175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:30.377456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0f7b5-e601-4e8e-ba33-2ae727a62c18 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:30.426158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e312f-0ab8-43ab-9bd4-b08d3b9604ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:30.427277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e312f-0ab8-43ab-9bd4-b08d3b9604ca None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:30.427499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e312f-0ab8-43ab-9bd4-b08d3b9604ca None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:30.427774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e312f-0ab8-43ab-9bd4-b08d3b9604ca None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209/users/2731105fbde540beb2907be40540fec5/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:30.432171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e312f-0ab8-43ab-9bd4-b08d3b9604ca None admin] RBAC: Authorizing `identity:create_grant(project_id=47aa8e95728c4c20aed763feb0b5a209, user_id=2731105fbde540beb2907be40540fec5, role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:30.434573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e312f-0ab8-43ab-9bd4-b08d3b9604ca None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:30.457083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 565/1123] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:43:30 2022] PUT /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209/users/2731105fbde540beb2907be40540fec5/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:43:32.755407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf484b5-b139-41d1-8a41-91c4dc03c5c3 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:32.755960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf484b5-b139-41d1-8a41-91c4dc03c5c3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:32.755960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf484b5-b139-41d1-8a41-91c4dc03c5c3 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:32.757357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 559/1124] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:32.763380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8b890-2e51-47d4-877c-7306148b0732 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:32.764008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8b890-2e51-47d4-877c-7306148b0732 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:32.764008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8b890-2e51-47d4-877c-7306148b0732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:32.782969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f8f8b890-2e51-47d4-877c-7306148b0732 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:32.784198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f8f8b890-2e51-47d4-877c-7306148b0732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:32.885301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 566/1125] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:32.917805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf484b5-b139-41d1-8a41-91c4dc03c5c3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:33.074588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf484b5-b139-41d1-8a41-91c4dc03c5c3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:33.130048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:33.130921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:33.131207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:33.131485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] PATH_INFO: `/v3/roles/c83e8b6745ad4b8ca12e7e6462784692` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:33.133237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] RBAC: Authorizing `identity:get_role(role_id=c83e8b6745ad4b8ca12e7e6462784692)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:33.134621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:33.136317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 560/1126] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:32 2022] GET /identity/v3/roles/c83e8b6745ad4b8ca12e7e6462784692 => generated 223 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:33.144081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8b890-2e51-47d4-877c-7306148b0732 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:33.184175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:33.276871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8b890-2e51-47d4-877c-7306148b0732 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:33.331414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:33.332220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:33.332511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:33.332791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] PATH_INFO: `/v3/users/2731105fbde540beb2907be40540fec5` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:33.334195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] RBAC: Authorizing `identity:get_user(user_id=2731105fbde540beb2907be40540fec5)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:33.335685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:33.337498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 567/1127] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:33 2022] GET /identity/v3/users/2731105fbde540beb2907be40540fec5 => generated 286 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:33.343815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:33.363041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:33.474402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7800a46-5eff-4a14-8066-a20368c1463d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:33.523422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:33.524241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:33.524573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:33.524932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:33.625889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:33.627354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:33.644951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:33.762672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 561/1128] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:43:33.771445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:33.921451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:33.933759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a906280-9754-4c75-8698-3164971dccc4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:33.974146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739317f9-3c78-4071-96e9-84291a494ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:33.975568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739317f9-3c78-4071-96e9-84291a494ab7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:33.975568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739317f9-3c78-4071-96e9-84291a494ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:33.975568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739317f9-3c78-4071-96e9-84291a494ab7 None admin] PATH_INFO: `/v3/projects/47aa8e95728c4c20aed763feb0b5a209` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:33.976955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739317f9-3c78-4071-96e9-84291a494ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=47aa8e95728c4c20aed763feb0b5a209)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:33.978423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739317f9-3c78-4071-96e9-84291a494ab7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:33.980214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 568/1129] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:33 2022] GET /identity/v3/projects/47aa8e95728c4c20aed763feb0b5a209 => generated 311 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:33.986824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:34.144080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098249f9-ef56-4a0d-8e4d-816485d09014 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:34.160476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:34.185399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d4096b-da67-4807-af20-e63e585d4bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:34.186145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4096b-da67-4807-af20-e63e585d4bad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:34.186425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4096b-da67-4807-af20-e63e585d4bad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:34.186692 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d4096b-da67-4807-af20-e63e585d4bad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:34.187540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4096b-da67-4807-af20-e63e585d4bad None admin] RBAC: Adding query filter params (role.id=c83e8b6745ad4b8ca12e7e6462784692, scope.project.id=47aa8e95728c4c20aed763feb0b5a209, user.id=2731105fbde540beb2907be40540fec5) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:34.187850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4096b-da67-4807-af20-e63e585d4bad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:34.189944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d4096b-da67-4807-af20-e63e585d4bad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:34.197367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 562/1130] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:33 2022] GET /identity/v3/role_assignments?user.id=2731105fbde540beb2907be40540fec5&scope.project.id=47aa8e95728c4c20aed763feb0b5a209&role.id=c83e8b6745ad4b8ca12e7e6462784692 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:36.583945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f22935-3aa0-467c-a2a0-1399caed0a55 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:36.584786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f22935-3aa0-467c-a2a0-1399caed0a55 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:36.584786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f22935-3aa0-467c-a2a0-1399caed0a55 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:36.586130 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 569/1131] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:36.592332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb8436-e3f3-42c3-aeb1-c70f0b89550d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:36.593233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb8436-e3f3-42c3-aeb1-c70f0b89550d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:36.593233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb8436-e3f3-42c3-aeb1-c70f0b89550d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:36.612253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1ebb8436-e3f3-42c3-aeb1-c70f0b89550d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:36.613802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1ebb8436-e3f3-42c3-aeb1-c70f0b89550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:36.698207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 563/1132] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:36.723108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f22935-3aa0-467c-a2a0-1399caed0a55 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:36.900213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f22935-3aa0-467c-a2a0-1399caed0a55 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:36.948914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e35fc-d911-4848-a85e-9a2f6d055e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:36.949791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e35fc-d911-4848-a85e-9a2f6d055e0f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:36.950118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e35fc-d911-4848-a85e-9a2f6d055e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:36.950440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e35fc-d911-4848-a85e-9a2f6d055e0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:36.951902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e35fc-d911-4848-a85e-9a2f6d055e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:36.953513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e35fc-d911-4848-a85e-9a2f6d055e0f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:36.955907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 570/1133] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:36.962712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb8436-e3f3-42c3-aeb1-c70f0b89550d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:37.143709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb8436-e3f3-42c3-aeb1-c70f0b89550d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:37.160430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:37.212804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8ea83-b780-4eb2-95bb-ce081dc95a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:37.213970 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8ea83-b780-4eb2-95bb-ce081dc95a06 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:37.214453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8ea83-b780-4eb2-95bb-ce081dc95a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:37.214935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8ea83-b780-4eb2-95bb-ce081dc95a06 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:37.216277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8ea83-b780-4eb2-95bb-ce081dc95a06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:37.218667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8ea83-b780-4eb2-95bb-ce081dc95a06 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:37.266947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 564/1134] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:43:36 2022] POST /identity/v3/projects => generated 311 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:43:40.016097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7470-065f-4332-a01b-b3c1810ad6ff None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:40.017691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7470-065f-4332-a01b-b3c1810ad6ff None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:40.017874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b7470-065f-4332-a01b-b3c1810ad6ff None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:40.019855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 571/1135] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:40.027131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ae0dc-1526-46ab-9ae5-93b266e8d186 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:40.027726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ae0dc-1526-46ab-9ae5-93b266e8d186 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:40.028077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ae0dc-1526-46ab-9ae5-93b266e8d186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:40.057220 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ab5ae0dc-1526-46ab-9ae5-93b266e8d186 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:40.059342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ab5ae0dc-1526-46ab-9ae5-93b266e8d186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:40.184035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 565/1136] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:40.217981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b7470-065f-4332-a01b-b3c1810ad6ff None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:40.258944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:40.378420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b7470-065f-4332-a01b-b3c1810ad6ff None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:40.422025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4066a-ad5e-4a72-ab3d-8a1ee4782ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:40.423416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4066a-ad5e-4a72-ab3d-8a1ee4782ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:40.423704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4066a-ad5e-4a72-ab3d-8a1ee4782ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:40.423980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4066a-ad5e-4a72-ab3d-8a1ee4782ac4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:40.425377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4066a-ad5e-4a72-ab3d-8a1ee4782ac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:40.426857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4066a-ad5e-4a72-ab3d-8a1ee4782ac4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:40.428590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 572/1137] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:43:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:40.435628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ae0dc-1526-46ab-9ae5-93b266e8d186 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:40.607189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:40.631630 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ae0dc-1526-46ab-9ae5-93b266e8d186 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:40.684751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdec0be-d553-4cc8-962d-4890edd80de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:40.686056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec0be-d553-4cc8-962d-4890edd80de1 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:40.686276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec0be-d553-4cc8-962d-4890edd80de1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:40.686585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec0be-d553-4cc8-962d-4890edd80de1 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:40.687344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdec0be-d553-4cc8-962d-4890edd80de1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:40.689506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdec0be-d553-4cc8-962d-4890edd80de1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:40.716921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 566/1138] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:43:40 2022] POST /identity/v3/users => generated 286 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:43:43.133430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5f506-9b52-4e85-b914-b2e2d4ce96e9 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:43.134264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5f506-9b52-4e85-b914-b2e2d4ce96e9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:43.134264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5f506-9b52-4e85-b914-b2e2d4ce96e9 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:43.135305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 573/1139] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:43:43.143700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cfa3c-a847-4163-bee2-dbaa5d5a3ca4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:43.144570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cfa3c-a847-4163-bee2-dbaa5d5a3ca4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:43.145058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cfa3c-a847-4163-bee2-dbaa5d5a3ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:43.176388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-190cfa3c-a847-4163-bee2-dbaa5d5a3ca4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:43.178590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-190cfa3c-a847-4163-bee2-dbaa5d5a3ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:43.314754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 567/1140] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:43.339539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5f506-9b52-4e85-b914-b2e2d4ce96e9 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:43.478347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:43.488081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5f506-9b52-4e85-b914-b2e2d4ce96e9 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:43.535739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:43.536764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:43.537202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:43.537575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:43.551319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:43.559718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:43.561993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:43.563614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 574/1141] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:43:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:43.572484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cfa3c-a847-4163-bee2-dbaa5d5a3ca4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:43.678161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:43.716711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cfa3c-a847-4163-bee2-dbaa5d5a3ca4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:43.762905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:43.763507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:43.763790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:43.764065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:43.764832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:43.765172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:43.767215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:43.780120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 568/1142] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:43:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:43.790001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:43.964787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c353-aa4b-4730-b24a-98fdb0dbd622 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:44.006465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:44.007362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:44.007644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:44.007912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] PATH_INFO: `/v3/users/e1fb7619f3dd4215a7ca0f452c52e6ac` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:44.024804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fb7619f3dd4215a7ca0f452c52e6ac)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:44.026338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:44.028150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 575/1143] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:43 2022] GET /identity/v3/users/e1fb7619f3dd4215a7ca0f452c52e6ac => generated 286 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:44.036139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:44.166714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fde8da-2eae-48b8-8a17-0cc3b9e7cd63 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:44.223376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:44.224546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:44.225089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:44.225609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:44.232602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:44.324825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:44.326373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:44.484153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 569/1144] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:43:44.492392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:44.518561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:44.637675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822822ff-2f07-4c57-8dce-47626dded985 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:44.686667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e396c-d5b5-4b56-889c-45b1ea8a6130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:44.688182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e396c-d5b5-4b56-889c-45b1ea8a6130 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:44.688670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e396c-d5b5-4b56-889c-45b1ea8a6130 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:44.689195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e396c-d5b5-4b56-889c-45b1ea8a6130 None admin] PATH_INFO: `/v3/projects/e591ce719a4347e497e895bd260c34fd` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:44.691565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e396c-d5b5-4b56-889c-45b1ea8a6130 None admin] RBAC: Authorizing `identity:get_project(project_id=e591ce719a4347e497e895bd260c34fd)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:44.694380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e396c-d5b5-4b56-889c-45b1ea8a6130 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:44.698030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 576/1145] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:44 2022] GET /identity/v3/projects/e591ce719a4347e497e895bd260c34fd => generated 311 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:44.707833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:44.753709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:44.846972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f52f9-21dd-42da-92b8-6d4e7b55886a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:44.894776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbf241-f0c5-4462-b934-33cb3002cd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:44.895780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbf241-f0c5-4462-b934-33cb3002cd6a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:44.896229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbf241-f0c5-4462-b934-33cb3002cd6a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:44.896702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbf241-f0c5-4462-b934-33cb3002cd6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:44.898384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbf241-f0c5-4462-b934-33cb3002cd6a None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=e591ce719a4347e497e895bd260c34fd, user.id=e1fb7619f3dd4215a7ca0f452c52e6ac) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:44.898925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbf241-f0c5-4462-b934-33cb3002cd6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:44.901704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbf241-f0c5-4462-b934-33cb3002cd6a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:44.911771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 570/1146] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:44 2022] GET /identity/v3/role_assignments?user.id=e1fb7619f3dd4215a7ca0f452c52e6ac&scope.project.id=e591ce719a4347e497e895bd260c34fd&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:47.295890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3026f7-b6f3-4391-be59-611e2bd9d331 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:47.296492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3026f7-b6f3-4391-be59-611e2bd9d331 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:47.296929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3026f7-b6f3-4391-be59-611e2bd9d331 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:47.299648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 577/1147] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:47.307611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a62cb-5dcb-4499-b753-3d8b5e7d595f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:47.308364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a62cb-5dcb-4499-b753-3d8b5e7d595f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:47.308364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a62cb-5dcb-4499-b753-3d8b5e7d595f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:47.327599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-be8a62cb-5dcb-4499-b753-3d8b5e7d595f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:47.329567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-be8a62cb-5dcb-4499-b753-3d8b5e7d595f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:47.414972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 571/1148] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:47.439613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3026f7-b6f3-4391-be59-611e2bd9d331 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:47.558722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:47.583064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3026f7-b6f3-4391-be59-611e2bd9d331 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:47.629407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:47.630383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:47.630768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:47.631150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:47.645004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:47.653452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:47.655313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:47.657007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 578/1149] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:43:47.665360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a62cb-5dcb-4499-b753-3d8b5e7d595f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:47.706380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:47.819605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a62cb-5dcb-4499-b753-3d8b5e7d595f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:47.886734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:47.887492 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:47.887817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:47.888132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:47.889011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:47.889365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:47.890823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:47.901938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 572/1150] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:47.908712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:48.086243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f8cc9-ac71-4286-bda0-ca70393b28dd None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:48.151648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:48.153251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:48.153750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:48.154260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] PATH_INFO: `/v3/users/e1fb7619f3dd4215a7ca0f452c52e6ac` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:48.156725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fb7619f3dd4215a7ca0f452c52e6ac)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:48.160143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:48.163513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 579/1151] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:47 2022] GET /identity/v3/users/e1fb7619f3dd4215a7ca0f452c52e6ac => generated 286 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:48.171324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:48.290011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:48.300925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab574f-8df8-4f85-a8d2-af1eaf9419f5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:48.343695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be5e46-8854-4d2b-92d1-74c2dd1c0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:48.344600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5e46-8854-4d2b-92d1-74c2dd1c0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:48.344916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5e46-8854-4d2b-92d1-74c2dd1c0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:48.345194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5e46-8854-4d2b-92d1-74c2dd1c0cd1 None admin] PATH_INFO: `/v3/projects/e591ce719a4347e497e895bd260c34fd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:48.346567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be5e46-8854-4d2b-92d1-74c2dd1c0cd1 None admin] RBAC: Authorizing `identity:get_project(project_id=e591ce719a4347e497e895bd260c34fd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:48.348054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be5e46-8854-4d2b-92d1-74c2dd1c0cd1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:48.350104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 573/1152] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:48 2022] GET /identity/v3/projects/e591ce719a4347e497e895bd260c34fd => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:48.357161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:48.374494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:48.508486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d389e8-9ef1-463a-bbcd-6467a6d3a783 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:48.553178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ac793-f3f1-4d1b-9316-77faa2b47626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:48.554108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ac793-f3f1-4d1b-9316-77faa2b47626 None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:48.554389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ac793-f3f1-4d1b-9316-77faa2b47626 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:48.554663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ac793-f3f1-4d1b-9316-77faa2b47626 None admin] PATH_INFO: `/v3/projects/e591ce719a4347e497e895bd260c34fd/users/e1fb7619f3dd4215a7ca0f452c52e6ac/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:48.557022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ac793-f3f1-4d1b-9316-77faa2b47626 None admin] RBAC: Authorizing `identity:create_grant(project_id=e591ce719a4347e497e895bd260c34fd, user_id=e1fb7619f3dd4215a7ca0f452c52e6ac, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:48.558415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ac793-f3f1-4d1b-9316-77faa2b47626 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:48.575985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 580/1153] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:43:48 2022] PUT /identity/v3/projects/e591ce719a4347e497e895bd260c34fd/users/e1fb7619f3dd4215a7ca0f452c52e6ac/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:43:50.883345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1e69-69c1-47b2-8262-d30c7fd1a361 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:50.883345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1e69-69c1-47b2-8262-d30c7fd1a361 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:50.884155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1e69-69c1-47b2-8262-d30c7fd1a361 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:50.885083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 574/1154] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:50.892349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d6330-dc0a-4812-8e1b-61ec80bded6c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:50.892980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d6330-dc0a-4812-8e1b-61ec80bded6c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:50.893387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d6330-dc0a-4812-8e1b-61ec80bded6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:50.913545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-628d6330-dc0a-4812-8e1b-61ec80bded6c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:50.914866 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-628d6330-dc0a-4812-8e1b-61ec80bded6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:50.992570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:50.998664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 581/1155] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:51.024616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d1e69-69c1-47b2-8262-d30c7fd1a361 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:51.232419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d1e69-69c1-47b2-8262-d30c7fd1a361 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:51.276074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:51.276904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:51.277189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:51.277458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:51.293214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:43:51.301434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:51.303596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:43:51.305758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 575/1156] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:43:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:51.313068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d6330-dc0a-4812-8e1b-61ec80bded6c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:51.511088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d6330-dc0a-4812-8e1b-61ec80bded6c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:51.559987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:51.560692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:51.561024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:51.561313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:51.562108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:51.562426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:51.563896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:51.574263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 582/1157] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:43:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:51.581284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:51.613714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:51.721989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9377af-f573-48c5-bebe-3f11463921a0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:51.767985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:51.768801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:51.769382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:51.769662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] PATH_INFO: `/v3/users/e1fb7619f3dd4215a7ca0f452c52e6ac` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:51.771012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] RBAC: Authorizing `identity:get_user(user_id=e1fb7619f3dd4215a7ca0f452c52e6ac)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:51.772487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:51.774307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 576/1158] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:43:51 2022] GET /identity/v3/users/e1fb7619f3dd4215a7ca0f452c52e6ac => generated 286 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:43:51.781023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:51.858946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:51.911964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fd136-c50e-47f3-9c7d-cc944563ffbb None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:51.959825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:51.960547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:51.960899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:51.961231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:52.052495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:52.055498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:52.139524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:52.198951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 583/1159] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:43:52.205597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:52.325245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:52.341387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dfa2-6ff3-475f-a7bb-abf8be1baf98 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:52.386640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be9a8b-4b81-4367-a98f-ba05ec8145b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:52.387517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be9a8b-4b81-4367-a98f-ba05ec8145b9 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:52.387794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be9a8b-4b81-4367-a98f-ba05ec8145b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:52.388069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be9a8b-4b81-4367-a98f-ba05ec8145b9 None admin] PATH_INFO: `/v3/projects/e591ce719a4347e497e895bd260c34fd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:52.389333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be9a8b-4b81-4367-a98f-ba05ec8145b9 None admin] RBAC: Authorizing `identity:get_project(project_id=e591ce719a4347e497e895bd260c34fd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:52.391060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be9a8b-4b81-4367-a98f-ba05ec8145b9 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:52.393423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 577/1160] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:43:52 2022] GET /identity/v3/projects/e591ce719a4347e497e895bd260c34fd => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:52.400885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:52.554281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc007f9-77b2-44bf-bfc9-242a0373a563 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:52.599899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d458c-8aba-4a2f-8349-2b0943ae7b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:52.600468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d458c-8aba-4a2f-8349-2b0943ae7b9a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:52.600749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d458c-8aba-4a2f-8349-2b0943ae7b9a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:52.601170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d458c-8aba-4a2f-8349-2b0943ae7b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:52.602155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d458c-8aba-4a2f-8349-2b0943ae7b9a None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=e591ce719a4347e497e895bd260c34fd, user.id=e1fb7619f3dd4215a7ca0f452c52e6ac) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:52.602473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d458c-8aba-4a2f-8349-2b0943ae7b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:52.603984 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d458c-8aba-4a2f-8349-2b0943ae7b9a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:52.619898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 584/1161] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:43:52 2022] GET /identity/v3/role_assignments?user.id=e1fb7619f3dd4215a7ca0f452c52e6ac&scope.project.id=e591ce719a4347e497e895bd260c34fd&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:54.991515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b89dd-a84b-459b-8424-f1da49f21602 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:54.991515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b89dd-a84b-459b-8424-f1da49f21602 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:54.992399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b89dd-a84b-459b-8424-f1da49f21602 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:54.993646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 578/1162] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:54.999756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fe758-c772-4d17-90ad-0ee438d09a2c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:55.000358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fe758-c772-4d17-90ad-0ee438d09a2c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:55.000358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fe758-c772-4d17-90ad-0ee438d09a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:55.020165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f89fe758-c772-4d17-90ad-0ee438d09a2c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:55.021586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f89fe758-c772-4d17-90ad-0ee438d09a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:55.049059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:55.107404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 585/1163] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:43:55.135320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b89dd-a84b-459b-8424-f1da49f21602 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:55.296266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b89dd-a84b-459b-8424-f1da49f21602 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:55.341375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:55.342044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:55.342329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:55.342613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:55.374151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:55.427867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:55.429478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:55.573295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 579/1164] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:43:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:43:55.579684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fe758-c772-4d17-90ad-0ee438d09a2c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:55.758608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fe758-c772-4d17-90ad-0ee438d09a2c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:55.799561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285549b-91de-44c8-a8de-b990f09d66ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:55.800383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285549b-91de-44c8-a8de-b990f09d66ec None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:55.800673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285549b-91de-44c8-a8de-b990f09d66ec None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:55.800991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285549b-91de-44c8-a8de-b990f09d66ec None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:55.809182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285549b-91de-44c8-a8de-b990f09d66ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:55.811547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285549b-91de-44c8-a8de-b990f09d66ec None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4285549b-91de-44c8-a8de-b990f09d66ec None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:43:55.827836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:43:55.830553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:43:55.832140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 586/1165] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:43:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:43:55.839302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:55.906623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:55.959063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2257d8-4c43-4fac-9782-63b0387fd475 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:56.011664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb256b-e31d-4af4-a97e-49feff844b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:56.012703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb256b-e31d-4af4-a97e-49feff844b73 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:56.013222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb256b-e31d-4af4-a97e-49feff844b73 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:56.013703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb256b-e31d-4af4-a97e-49feff844b73 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:56.014994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb256b-e31d-4af4-a97e-49feff844b73 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:43:56.015519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb256b-e31d-4af4-a97e-49feff844b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:56.018266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb256b-e31d-4af4-a97e-49feff844b73 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:56.027304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 580/1166] 10.222.0.72 () {56 vars in 1115 bytes} [Thu Jun 16 14:43:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:43:58.361219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e8a0-b69e-4355-b400-ab31355edead None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:58.361853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e8a0-b69e-4355-b400-ab31355edead None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:58.361853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e8a0-b69e-4355-b400-ab31355edead None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:58.363410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 587/1167] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:43:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:43:58.369848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fef1d-1288-4966-a333-1fee413d8fd0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:58.370514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fef1d-1288-4966-a333-1fee413d8fd0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:58.370514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fef1d-1288-4966-a333-1fee413d8fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:58.390961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f12fef1d-1288-4966-a333-1fee413d8fd0 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:43:58.392236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f12fef1d-1288-4966-a333-1fee413d8fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:43:58.480123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 581/1168] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:43:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:43:58.507139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107e8a0-b69e-4355-b400-ab31355edead None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:43:58.530097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:43:58.649252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107e8a0-b69e-4355-b400-ab31355edead None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:43:58.695628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611ed9e-1868-4374-8b03-c1741ef8abc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:43:58.696374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611ed9e-1868-4374-8b03-c1741ef8abc9 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:43:58.696706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611ed9e-1868-4374-8b03-c1741ef8abc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:43:58.697073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611ed9e-1868-4374-8b03-c1741ef8abc9 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:43:58.697903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611ed9e-1868-4374-8b03-c1741ef8abc9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:43:58.699400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611ed9e-1868-4374-8b03-c1741ef8abc9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:43:58.720138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 588/1169] 10.222.0.72 () {60 vars in 1138 bytes} [Thu Jun 16 14:43:58 2022] POST /identity/v3/domains => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:44:01.186474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0133b-8bf4-4bfb-9a34-9f0c117f8ee7 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:01.187028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0133b-8bf4-4bfb-9a34-9f0c117f8ee7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:01.187028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0133b-8bf4-4bfb-9a34-9f0c117f8ee7 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:01.189031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 582/1170] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:44:01.195305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b610715-3d0b-4cd0-905c-ce5c9bae2496 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:01.195937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b610715-3d0b-4cd0-905c-ce5c9bae2496 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:01.195937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b610715-3d0b-4cd0-905c-ce5c9bae2496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:01.214097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-3b610715-3d0b-4cd0-905c-ce5c9bae2496 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:44:01.215343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3b610715-3d0b-4cd0-905c-ce5c9bae2496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:44:01.304491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 589/1171] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:44:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:44:01.330766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0133b-8bf4-4bfb-9a34-9f0c117f8ee7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:01.393698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:01.494566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0133b-8bf4-4bfb-9a34-9f0c117f8ee7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:01.567723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadd517-2053-485f-b8aa-49a092cf719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:01.569278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadd517-2053-485f-b8aa-49a092cf719d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:01.569788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadd517-2053-485f-b8aa-49a092cf719d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:01.570298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadd517-2053-485f-b8aa-49a092cf719d None admin] PATH_INFO: `/v3/domains/ce96b9d25e894c2a9ee3c7b3aee45754` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:01.591541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadd517-2053-485f-b8aa-49a092cf719d None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce96b9d25e894c2a9ee3c7b3aee45754)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:01.595116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadd517-2053-485f-b8aa-49a092cf719d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:01.600426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 583/1172] 10.222.0.72 () {56 vars in 1183 bytes} [Thu Jun 16 14:44:01 2022] GET /identity/v3/domains/ce96b9d25e894c2a9ee3c7b3aee45754 => generated 267 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:01.610739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b610715-3d0b-4cd0-905c-ce5c9bae2496 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:01.627393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:01.771109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b610715-3d0b-4cd0-905c-ce5c9bae2496 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:01.821293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc517e-a825-445c-a1a5-496de960ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:01.822461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc517e-a825-445c-a1a5-496de960ddad None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:01.822944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc517e-a825-445c-a1a5-496de960ddad None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:01.823423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc517e-a825-445c-a1a5-496de960ddad None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:01.824882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc517e-a825-445c-a1a5-496de960ddad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:01.827792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc517e-a825-445c-a1a5-496de960ddad None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:01.874167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 590/1173] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:44:01 2022] POST /identity/v3/projects => generated 361 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:44:04.332108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90acafd-2485-4111-9a6f-9685de04ddbd None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:04.332915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90acafd-2485-4111-9a6f-9685de04ddbd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:04.332915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90acafd-2485-4111-9a6f-9685de04ddbd None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:04.334263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 584/1174] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:44:04.341504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d09d3-113f-4fe4-aa42-7499c074d6d4 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:04.342100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d09d3-113f-4fe4-aa42-7499c074d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:04.342100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d09d3-113f-4fe4-aa42-7499c074d6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:04.363857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8e1d09d3-113f-4fe4-aa42-7499c074d6d4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:44:04.365684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8e1d09d3-113f-4fe4-aa42-7499c074d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:44:04.437232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:04.457397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 591/1175] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:44:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:44:04.484248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90acafd-2485-4111-9a6f-9685de04ddbd None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:04.641169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90acafd-2485-4111-9a6f-9685de04ddbd None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:04.655865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:04.695297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892467c-cefb-4baa-b39a-8b62937b2769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:04.696722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892467c-cefb-4baa-b39a-8b62937b2769 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:04.697203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892467c-cefb-4baa-b39a-8b62937b2769 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:04.697672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892467c-cefb-4baa-b39a-8b62937b2769 None admin] PATH_INFO: `/v3/domains/ce96b9d25e894c2a9ee3c7b3aee45754` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:04.699795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892467c-cefb-4baa-b39a-8b62937b2769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ce96b9d25e894c2a9ee3c7b3aee45754)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:04.702328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892467c-cefb-4baa-b39a-8b62937b2769 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:04.705893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 585/1176] 10.222.0.72 () {56 vars in 1183 bytes} [Thu Jun 16 14:44:04 2022] GET /identity/v3/domains/ce96b9d25e894c2a9ee3c7b3aee45754 => generated 267 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:04.715731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d09d3-113f-4fe4-aa42-7499c074d6d4 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:04.892704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d09d3-113f-4fe4-aa42-7499c074d6d4 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:04.940421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435545fe-f9cb-4407-9fd0-868d919b89f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:04.941214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435545fe-f9cb-4407-9fd0-868d919b89f0 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:04.941499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435545fe-f9cb-4407-9fd0-868d919b89f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:04.941848 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435545fe-f9cb-4407-9fd0-868d919b89f0 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:04.942711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435545fe-f9cb-4407-9fd0-868d919b89f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:04.945393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435545fe-f9cb-4407-9fd0-868d919b89f0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:04.979232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 592/1177] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:44:04 2022] POST /identity/v3/users => generated 311 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:44:07.394172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f24235-7c97-4400-ba39-73ee2ce367ba None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:07.395199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f24235-7c97-4400-ba39-73ee2ce367ba None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:07.395199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f24235-7c97-4400-ba39-73ee2ce367ba None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:07.396712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 586/1178] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:44:07.404302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e699bb-9aa7-4337-8ec6-9972d1cf12a6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:07.404939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e699bb-9aa7-4337-8ec6-9972d1cf12a6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:07.405306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e699bb-9aa7-4337-8ec6-9972d1cf12a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:07.433693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-56e699bb-9aa7-4337-8ec6-9972d1cf12a6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:44:07.435776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-56e699bb-9aa7-4337-8ec6-9972d1cf12a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:44:07.514579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:07.567044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 593/1179] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:44:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:44:07.604455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f24235-7c97-4400-ba39-73ee2ce367ba None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:07.769159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f24235-7c97-4400-ba39-73ee2ce367ba None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:07.813732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16741240-6c34-49a0-b741-508670245eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:07.814844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16741240-6c34-49a0-b741-508670245eca None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:07.815310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16741240-6c34-49a0-b741-508670245eca None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:07.815777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16741240-6c34-49a0-b741-508670245eca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:07.830163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16741240-6c34-49a0-b741-508670245eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-16741240-6c34-49a0-b741-508670245eca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:44:07.838611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:44:07.840665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:44:07.842578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 587/1180] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:44:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:44:07.848494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e699bb-9aa7-4337-8ec6-9972d1cf12a6 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:07.987430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e699bb-9aa7-4337-8ec6-9972d1cf12a6 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:08.047895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:08.048723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:08.049139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:08.049514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:08.050449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:44:08.050851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:08.052367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:08.062664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 594/1181] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:44:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:08.069956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16741240-6c34-49a0-b741-508670245eca None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:08.133770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:08.231257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16741240-6c34-49a0-b741-508670245eca None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:08.276558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:08.277568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:08.277853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:08.278130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] PATH_INFO: `/v3/users/09c1302661ce4d119d5a1158b69719b9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:08.295883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] RBAC: Authorizing `identity:get_user(user_id=09c1302661ce4d119d5a1158b69719b9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:08.298164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:08.299973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 588/1182] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:44:08 2022] GET /identity/v3/users/09c1302661ce4d119d5a1158b69719b9 => generated 311 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:08.307557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:08.351422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:08.436729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbf568-774a-4572-9f99-ed75db226de2 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:08.476531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:08.477254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:08.477545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:08.477828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:08.557025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:08.558608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:08.613067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:08.698371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 595/1183] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:44:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:44:08.708895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:08.843064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:08.854804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78f5ba-e81e-41cd-934b-a1ecf2c7eb69 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:08.923541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6e22f-682c-4fd4-8053-2f7ac68d85f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:08.925091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6e22f-682c-4fd4-8053-2f7ac68d85f7 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:08.925602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6e22f-682c-4fd4-8053-2f7ac68d85f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:08.926108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6e22f-682c-4fd4-8053-2f7ac68d85f7 None admin] PATH_INFO: `/v3/projects/6d3baa29661248569fa85144d8be29ef` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:08.928407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6e22f-682c-4fd4-8053-2f7ac68d85f7 None admin] RBAC: Authorizing `identity:get_project(project_id=6d3baa29661248569fa85144d8be29ef)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:08.931266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6e22f-682c-4fd4-8053-2f7ac68d85f7 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:08.934704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 589/1184] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:44:08 2022] GET /identity/v3/projects/6d3baa29661248569fa85144d8be29ef => generated 361 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:08.943395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:09.066952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae07fb-99ae-4d8e-af66-54946510e183 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:09.118124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60dfbc-59ff-4cba-b3c7-97b26177fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:09.118821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60dfbc-59ff-4cba-b3c7-97b26177fa72 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:09.119161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60dfbc-59ff-4cba-b3c7-97b26177fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:09.119491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60dfbc-59ff-4cba-b3c7-97b26177fa72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:09.120498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60dfbc-59ff-4cba-b3c7-97b26177fa72 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=6d3baa29661248569fa85144d8be29ef, user.id=09c1302661ce4d119d5a1158b69719b9) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:44:09.120894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60dfbc-59ff-4cba-b3c7-97b26177fa72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:09.122497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60dfbc-59ff-4cba-b3c7-97b26177fa72 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:09.126350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:09.130671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 596/1185] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:44:08 2022] GET /identity/v3/role_assignments?user.id=09c1302661ce4d119d5a1158b69719b9&scope.project.id=6d3baa29661248569fa85144d8be29ef&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:11.487089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76335-2dc5-46cd-b7d4-cc6eda1370cd None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:11.487878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76335-2dc5-46cd-b7d4-cc6eda1370cd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:11.487878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76335-2dc5-46cd-b7d4-cc6eda1370cd None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:11.489255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 590/1186] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:44:11.495407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383df25-9c6b-4956-8666-faa6c3d49f0a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:11.495971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383df25-9c6b-4956-8666-faa6c3d49f0a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:11.495971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383df25-9c6b-4956-8666-faa6c3d49f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:11.513976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-9383df25-9c6b-4956-8666-faa6c3d49f0a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:44:11.515230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-9383df25-9c6b-4956-8666-faa6c3d49f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:44:11.600664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 597/1187] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:44:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:44:11.624859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76335-2dc5-46cd-b7d4-cc6eda1370cd None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:11.783173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76335-2dc5-46cd-b7d4-cc6eda1370cd None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:11.841888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:11.842906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:11.843282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:11.843670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:11.861446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:44:11.871725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:44:11.872814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:44:11.873717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 591/1188] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:44:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:44:11.881452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383df25-9c6b-4956-8666-faa6c3d49f0a None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:12.021217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383df25-9c6b-4956-8666-faa6c3d49f0a None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:12.069624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:12.077578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:12.078321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:12.078609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:12.078899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:12.079664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:44:12.079977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:12.083420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:12.091544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 598/1189] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:44:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:44:12.099379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:12.138318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:12.241625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea024-57e1-45f1-a37f-c4d10506b8e8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:12.292652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:12.294233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:12.294747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:12.295258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] PATH_INFO: `/v3/users/09c1302661ce4d119d5a1158b69719b9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:12.297568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] RBAC: Authorizing `identity:get_user(user_id=09c1302661ce4d119d5a1158b69719b9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:12.300378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:12.303867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 592/1190] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:44:12 2022] GET /identity/v3/users/09c1302661ce4d119d5a1158b69719b9 => generated 311 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:12.311409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:12.469101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c0d5e-d8c6-4839-97a5-dc74c6d7a380 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:12.520243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db3192-f327-488d-ac09-ef02bed44e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:12.521338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db3192-f327-488d-ac09-ef02bed44e38 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:12.521790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db3192-f327-488d-ac09-ef02bed44e38 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:12.522100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db3192-f327-488d-ac09-ef02bed44e38 None admin] PATH_INFO: `/v3/projects/6d3baa29661248569fa85144d8be29ef` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:12.523636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db3192-f327-488d-ac09-ef02bed44e38 None admin] RBAC: Authorizing `identity:get_project(project_id=6d3baa29661248569fa85144d8be29ef)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:12.525443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db3192-f327-488d-ac09-ef02bed44e38 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:12.527239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 599/1191] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:44:12 2022] GET /identity/v3/projects/6d3baa29661248569fa85144d8be29ef => generated 361 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:12.538280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:12.654650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:12.672366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04977b8-8957-49f6-962f-f8356583b911 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:12.713288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d02e31-1042-49a6-952d-9066d38afa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:12.714198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02e31-1042-49a6-952d-9066d38afa9d None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:12.714482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02e31-1042-49a6-952d-9066d38afa9d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:12.714760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d02e31-1042-49a6-952d-9066d38afa9d None admin] PATH_INFO: `/v3/projects/6d3baa29661248569fa85144d8be29ef/users/09c1302661ce4d119d5a1158b69719b9/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:12.716829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d02e31-1042-49a6-952d-9066d38afa9d None admin] RBAC: Authorizing `identity:create_grant(project_id=6d3baa29661248569fa85144d8be29ef, user_id=09c1302661ce4d119d5a1158b69719b9, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:12.718340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d02e31-1042-49a6-952d-9066d38afa9d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:12.735894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 593/1192] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:44:12 2022] PUT /identity/v3/projects/6d3baa29661248569fa85144d8be29ef/users/09c1302661ce4d119d5a1158b69719b9/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:44:15.191870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bf92d-0334-4307-9c78-ca85d1ed30aa None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:15.192469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bf92d-0334-4307-9c78-ca85d1ed30aa None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:15.192469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bf92d-0334-4307-9c78-ca85d1ed30aa None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:15.193996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 600/1193] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:44:15.200219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3df5e-bafd-4c93-8258-539963b25bf3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:15.200795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3df5e-bafd-4c93-8258-539963b25bf3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:15.200795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3df5e-bafd-4c93-8258-539963b25bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:15.220235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e6c3df5e-bafd-4c93-8258-539963b25bf3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:44:15.221587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e6c3df5e-bafd-4c93-8258-539963b25bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:44:15.303727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 594/1194] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:44:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:44:15.328358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bf92d-0334-4307-9c78-ca85d1ed30aa None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:15.400028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:15.475692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bf92d-0334-4307-9c78-ca85d1ed30aa None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:15.544997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:15.546390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:15.546918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:15.547514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:15.571035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:44:15.586516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:44:15.587743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:44:15.588831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 601/1195] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:44:15.596214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3df5e-bafd-4c93-8258-539963b25bf3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:15.725944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:15.748330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3df5e-bafd-4c93-8258-539963b25bf3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:15.792386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:15.793258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:15.793638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:15.794016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:15.794892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:44:15.795304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:15.796824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:15.807616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 595/1196] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:44:15.818045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:15.952051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409365-2ab6-4fa4-b372-4b3cd293ba83 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:15.990502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:16.018029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:16.019867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:16.020373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:16.021021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] PATH_INFO: `/v3/users/09c1302661ce4d119d5a1158b69719b9` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:16.023404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] RBAC: Authorizing `identity:get_user(user_id=09c1302661ce4d119d5a1158b69719b9)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:16.026476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:16.029943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 602/1197] 10.222.0.72 () {56 vars in 1177 bytes} [Thu Jun 16 14:44:15 2022] GET /identity/v3/users/09c1302661ce4d119d5a1158b69719b9 => generated 311 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:16.036012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:16.158670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a0472-744d-4916-a078-bb919c7c6a16 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:16.207875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:16.208982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:16.209483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:16.209951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:16.235723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:16.321517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:16.323445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:16.453822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 596/1198] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:44:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:44:16.463663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:16.649556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92dca-0727-4f84-853f-efc1acf69fc5 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:16.714372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc6c2c-7b91-459e-8e28-fa3aa3348b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:16.715693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc6c2c-7b91-459e-8e28-fa3aa3348b0d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:16.716169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc6c2c-7b91-459e-8e28-fa3aa3348b0d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:16.716606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc6c2c-7b91-459e-8e28-fa3aa3348b0d None admin] PATH_INFO: `/v3/projects/6d3baa29661248569fa85144d8be29ef` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:16.718546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc6c2c-7b91-459e-8e28-fa3aa3348b0d None admin] RBAC: Authorizing `identity:get_project(project_id=6d3baa29661248569fa85144d8be29ef)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:16.721749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc6c2c-7b91-459e-8e28-fa3aa3348b0d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:16.724670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 603/1199] 10.222.0.72 () {56 vars in 1186 bytes} [Thu Jun 16 14:44:16 2022] GET /identity/v3/projects/6d3baa29661248569fa85144d8be29ef => generated 361 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:16.731598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:44:16.803478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:16.928961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6be71-8623-4d28-98ae-0d100028e09c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:44:16.977291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b705748-34ee-4c92-bcab-2790b320ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:44:16.977964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b705748-34ee-4c92-bcab-2790b320ae9a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:44:16.978245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b705748-34ee-4c92-bcab-2790b320ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:44:16.978515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b705748-34ee-4c92-bcab-2790b320ae9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:44:16.979466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b705748-34ee-4c92-bcab-2790b320ae9a None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=6d3baa29661248569fa85144d8be29ef, user.id=09c1302661ce4d119d5a1158b69719b9) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:44:16.979947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b705748-34ee-4c92-bcab-2790b320ae9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:44:16.981482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b705748-34ee-4c92-bcab-2790b320ae9a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:44:16.989017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 597/1200] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:44:16 2022] GET /identity/v3/role_assignments?user.id=09c1302661ce4d119d5a1158b69719b9&scope.project.id=6d3baa29661248569fa85144d8be29ef&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:44:26.706796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=72 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:44:26.985964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=46 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:25.701801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e59784-eace-47ae-9c25-419f63bc7e6f None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:25.702773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e59784-eace-47ae-9c25-419f63bc7e6f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:25.702773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e59784-eace-47ae-9c25-419f63bc7e6f None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:25.705142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 604/1201] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:45:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:45:25.715690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e9f46-977f-47c7-8777-5ed38ad1399d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:25.716575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e9f46-977f-47c7-8777-5ed38ad1399d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:25.716687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e9f46-977f-47c7-8777-5ed38ad1399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:25.756056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2f7e9f46-977f-47c7-8777-5ed38ad1399d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:45:25.758608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2f7e9f46-977f-47c7-8777-5ed38ad1399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:45:25.917595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 598/1202] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:45:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:45:25.961390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e59784-eace-47ae-9c25-419f63bc7e6f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:26.182626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e59784-eace-47ae-9c25-419f63bc7e6f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:26.277317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:26.279163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:26.280352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:26.281108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:26.313636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:26.316668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:26.358252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:26.360182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:26.362325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:26.362325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:26.362325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:26.362325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:26.366029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 605/1203] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:26.377375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e9f46-977f-47c7-8777-5ed38ad1399d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:26.618418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e9f46-977f-47c7-8777-5ed38ad1399d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:26.674926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:26.690098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d242dd-b17e-4767-b87e-f9aad27623bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:26.691528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d242dd-b17e-4767-b87e-f9aad27623bc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:26.692060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d242dd-b17e-4767-b87e-f9aad27623bc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:26.692714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d242dd-b17e-4767-b87e-f9aad27623bc None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:26.694498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d242dd-b17e-4767-b87e-f9aad27623bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:26.695218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d242dd-b17e-4767-b87e-f9aad27623bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:26.698562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d242dd-b17e-4767-b87e-f9aad27623bc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:26.715617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 599/1204] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:26.724457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:26.837182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:26.911332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d48c5-fc62-49a0-b584-8ed22dea7670 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:26.972744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067704d-5e1f-49cc-b3b2-0437e5065c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:26.974280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067704d-5e1f-49cc-b3b2-0437e5065c71 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:26.974970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067704d-5e1f-49cc-b3b2-0437e5065c71 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:26.975463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067704d-5e1f-49cc-b3b2-0437e5065c71 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:26.976814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067704d-5e1f-49cc-b3b2-0437e5065c71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:26.979600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067704d-5e1f-49cc-b3b2-0437e5065c71 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:27.017924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 606/1205] 10.222.0.72 () {60 vars in 1132 bytes} [Thu Jun 16 14:45:26 2022] POST /identity/v3/users => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:45:31.000903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46e1b-36eb-46a9-8a5c-d479db4529c3 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:31.000903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46e1b-36eb-46a9-8a5c-d479db4529c3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:31.001924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46e1b-36eb-46a9-8a5c-d479db4529c3 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:31.003612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 600/1206] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:45:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:45:31.013738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53497c1-494c-4725-b5a5-fd9fc411a082 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:31.014615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53497c1-494c-4725-b5a5-fd9fc411a082 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:31.014615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53497c1-494c-4725-b5a5-fd9fc411a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:31.038892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-d53497c1-494c-4725-b5a5-fd9fc411a082 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:45:31.041454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d53497c1-494c-4725-b5a5-fd9fc411a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:45:31.196260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 607/1207] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:45:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:45:31.228232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd46e1b-36eb-46a9-8a5c-d479db4529c3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:31.445937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd46e1b-36eb-46a9-8a5c-d479db4529c3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:31.515826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:31.516647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:31.517049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:31.517212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:31.536058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:45:31.549089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:31.550837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:45:31.552533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:31.561619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 601/1208] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:45:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:31.562075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53497c1-494c-4725-b5a5-fd9fc411a082 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:31.642657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:31.780674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53497c1-494c-4725-b5a5-fd9fc411a082 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:31.849554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:31.851991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:31.851991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:31.851991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:31.851991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:31.852740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:31.855137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:31.868724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 608/1209] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:45:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:45:31.875332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:31.965509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:32.047981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0d17-4009-4f48-8ffb-2a4cccc7cfb8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:32.118812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:32.120725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:32.121250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:32.121749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:32.147422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:32.149198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:32.175990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:32.178101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:32.180145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:32.180145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:32.180145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:32.180145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:32.180145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 602/1210] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:32.185148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:32.419557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:32.427118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95a74-e4c1-4322-8911-ecb9c102e243 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:32.501094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:32.502100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:32.502511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:32.502957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:32.504107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:32.504629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:32.507298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:32.541333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 609/1211] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:32.543494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:32.769050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e93a-d00a-469b-b948-0406d02787ad None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:32.772925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:32.848488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:32.849417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:32.849702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:32.849977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:32.861479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:32.863801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:45:32.900412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:32.902494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:32.909612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 603/1212] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:32.910396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:33.143788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570de9-cc37-4b21-9c4c-919b3fafd692 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:33.221896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:33.223503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:33.226042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:33.226042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:33.226528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:33.227140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:33.231262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:33.288270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 610/1213] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Jun 16 14:45:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:33.298449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:33.484332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605519f-a97e-403c-8e7e-d58ee902d2bc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:33.559625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:33.561237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:33.561673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:33.562005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:33.631289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:33.698860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:33.702690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:33.933948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 604/1214] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:45:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 14:45:33.946253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:33.984730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:34.197164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08117bc2-da81-465c-8b00-9af792c8d832 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:34.261774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:34.263211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:34.263554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:34.264289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:34.282614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:34.284399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:34.313274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:34.315022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:34.321496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:34.321496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:34.321496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:34.321496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:34.321496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 611/1215] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:34.321881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:34.429828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:34.502045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb13117-8f57-4987-99fc-4af5da631701 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:34.574221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:34.575328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:34.577132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:34.577132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:34.577947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:34.578389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:34.579988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:34.612909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 605/1216] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:34.619585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:34.762787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:34.827095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41883-a142-42ce-8523-b1bb3a17fe44 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:34.901724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:34.903114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:34.904789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:34.904789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:34.916168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:34.918054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:45:34.938256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:34.940270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:34.942365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:34.944301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 612/1217] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:34.954575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:35.155562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d41bff-a95e-4eeb-8d58-f98f3976b17f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:35.214842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:35.248478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8b382-4ad9-42d2-9fa2-e79e2bb28c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:35.250099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b382-4ad9-42d2-9fa2-e79e2bb28c62 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:35.250664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b382-4ad9-42d2-9fa2-e79e2bb28c62 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:35.251190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b382-4ad9-42d2-9fa2-e79e2bb28c62 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:35.254661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8b382-4ad9-42d2-9fa2-e79e2bb28c62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:35.255226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8b382-4ad9-42d2-9fa2-e79e2bb28c62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:35.257223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8b382-4ad9-42d2-9fa2-e79e2bb28c62 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:35.280116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 606/1218] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:35.290620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:35.479367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac623b-f004-4ded-a968-d2f7cf3cd6ad None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:35.530010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:35.542840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a22c78-e8b2-4779-823a-c71393ec8d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:35.544279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a22c78-e8b2-4779-823a-c71393ec8d16 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:35.545597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a22c78-e8b2-4779-823a-c71393ec8d16 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:35.546287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a22c78-e8b2-4779-823a-c71393ec8d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:35.548232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a22c78-e8b2-4779-823a-c71393ec8d16 None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=848d858044ed43e69aa055491cc29d80) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:35.548796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a22c78-e8b2-4779-823a-c71393ec8d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:35.552046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a22c78-e8b2-4779-823a-c71393ec8d16 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:35.566988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 613/1219] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:45:35 2022] GET /identity/v3/role_assignments?user.id=848d858044ed43e69aa055491cc29d80&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:45:38.536685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf6c1-63b3-4595-ba90-226da28dd26b None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:38.537451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf6c1-63b3-4595-ba90-226da28dd26b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:38.538133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf6c1-63b3-4595-ba90-226da28dd26b None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:38.548654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 607/1220] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:45:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:45:38.551305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129e2d0-4c4d-4e08-bf56-3d1fb80a6932 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:38.552161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129e2d0-4c4d-4e08-bf56-3d1fb80a6932 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:38.552325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129e2d0-4c4d-4e08-bf56-3d1fb80a6932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:38.578625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c129e2d0-4c4d-4e08-bf56-3d1fb80a6932 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:45:38.580282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c129e2d0-4c4d-4e08-bf56-3d1fb80a6932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:45:38.729292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 614/1221] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:45:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:45:38.758976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf6c1-63b3-4595-ba90-226da28dd26b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:38.952714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf6c1-63b3-4595-ba90-226da28dd26b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:39.023811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:39.025100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:39.025514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:39.025897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:39.042463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:45:39.054279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:39.056345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:39.057431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 608/1222] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:39.063324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129e2d0-4c4d-4e08-bf56-3d1fb80a6932 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:39.232061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129e2d0-4c4d-4e08-bf56-3d1fb80a6932 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:39.271900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:39.296324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:39.297572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:39.298149 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:39.298733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:39.300378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:39.301040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:39.303925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:39.319231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 615/1223] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:45:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:39.327451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:39.409694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:39.512882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edd4e8-e791-4f7e-9473-c218b0703b6f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:39.569954 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:39.572916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:39.572916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:39.572916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:39.595689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:39.597649 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:39.625805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:39.628070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:39.629168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:39.629168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:39.629168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:39.629168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:39.629168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 609/1224] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:39.635965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:39.805437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25781d20-ce69-449a-8150-2b98b2b5d6f9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:39.870136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:39.871291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:39.871780 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:39.872961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:39.874338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:39.874790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:39.877557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:39.909820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 616/1225] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:39.916666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:40.100459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:40.116048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d62f8-ec52-49ec-9a07-afa4436fbd92 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:40.191588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:40.193337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:40.193858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:40.194384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:40.211355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:40.214123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:45:40.243095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:40.244537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:40.246014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 610/1226] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:40.256220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:40.344550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:40.442687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec3353-3c93-4451-ba61-2ece139ca4f6 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:40.503544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:40.504619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:40.505166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:40.505602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:40.507189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:40.507680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:40.509493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:40.539680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 617/1227] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Jun 16 14:45:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:40.548107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:40.717509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634a3f7-5cbc-49ee-bc56-cc788cbdb33d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:40.780764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:40.782601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:40.784889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:40.784889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:40.800587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:40.802380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:40.835590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:40.836798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:40.838668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:40.838668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:40.838668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:40.838668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:40.838668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 611/1228] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:40.848779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:41.028658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d10cd-6888-4481-a3c3-5fc3e8ead771 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:41.032946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:41.087194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:41.087932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:41.088218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:41.088573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:41.089440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:41.089755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:41.092575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:41.108900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 618/1229] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:41.118105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:41.133863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:41.293935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa0d0f-ed81-4db5-90eb-3e176e7fd6a4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:41.362575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:41.363404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:41.363698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:41.364076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:41.377495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:41.381106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:45:41.411686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:41.414047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:41.417619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 612/1230] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:41.427259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:41.592920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21294ca3-b330-4e02-994d-783e51531983 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:41.658134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5054e-bb3a-480b-8da0-1143f13c04f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:41.659209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5054e-bb3a-480b-8da0-1143f13c04f3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:41.659717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5054e-bb3a-480b-8da0-1143f13c04f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:41.660191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5054e-bb3a-480b-8da0-1143f13c04f3 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:41.661441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5054e-bb3a-480b-8da0-1143f13c04f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:41.662013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5054e-bb3a-480b-8da0-1143f13c04f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:41.665980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5054e-bb3a-480b-8da0-1143f13c04f3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:41.688200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 619/1231] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:41.695733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:41.811355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:41.874974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfcb49e-886f-4e5d-ad77-08bc94231e5d None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:41.921373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec9c5e-6010-49d5-b51d-75301822e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:41.923037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9c5e-6010-49d5-b51d-75301822e866 None admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:41.923037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9c5e-6010-49d5-b51d-75301822e866 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:41.923037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec9c5e-6010-49d5-b51d-75301822e866 None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/848d858044ed43e69aa055491cc29d80/roles/5a44c123efd54f14bc8e76881f748f8b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:41.949553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec9c5e-6010-49d5-b51d-75301822e866 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=848d858044ed43e69aa055491cc29d80, role_id=5a44c123efd54f14bc8e76881f748f8b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:41.952650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec9c5e-6010-49d5-b51d-75301822e866 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:41.975464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 613/1232] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:45:41 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/848d858044ed43e69aa055491cc29d80/roles/5a44c123efd54f14bc8e76881f748f8b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:45:44.557951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522d4b-af8e-4501-87b1-aa604125a4bc None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:44.558634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522d4b-af8e-4501-87b1-aa604125a4bc None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:44.559117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522d4b-af8e-4501-87b1-aa604125a4bc None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:44.562097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 620/1233] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:45:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:45:44.571727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eaf6b-d3d3-406e-a77a-0fc65b3a5795 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:44.572430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eaf6b-d3d3-406e-a77a-0fc65b3a5795 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:44.572668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48eaf6b-d3d3-406e-a77a-0fc65b3a5795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:44.609219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b48eaf6b-d3d3-406e-a77a-0fc65b3a5795 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:45:44.612382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b48eaf6b-d3d3-406e-a77a-0fc65b3a5795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:45:44.728121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 614/1234] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:45:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:45:44.774392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522d4b-af8e-4501-87b1-aa604125a4bc None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:44.822695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:44.969817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522d4b-af8e-4501-87b1-aa604125a4bc None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:45.022678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:45.024282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:45.024282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:45.024960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:45.040755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:45:45.050961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:45.053168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:45.054503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 621/1235] 10.222.0.72 () {56 vars in 1102 bytes} [Thu Jun 16 14:45:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:45.061399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eaf6b-d3d3-406e-a77a-0fc65b3a5795 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:45.146443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:45.269246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48eaf6b-d3d3-406e-a77a-0fc65b3a5795 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:45.331711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:45.332562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:45.332903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:45.333254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:45.333942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] RBAC: Adding query filter params (name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:45.334254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:45.335698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:45.346440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 615/1236] 10.222.0.72 () {56 vars in 1103 bytes} [Thu Jun 16 14:45:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:45.354252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:45.498839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:45.519987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba091b-7824-4ad5-b0b6-f68da05dc6cc None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:45.578337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:45.579709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:45.580205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:45.580702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:45.601568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:45.603622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:45.631677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:45.633767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:45.635939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:45.635939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:45.635939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:45.635939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:45.637075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 622/1237] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:45.643613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:45.828122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945a3d1-e8cc-4301-83da-ad2d8c5cbb35 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:45.842313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:45.880471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:45.881188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:45.881472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:45.881781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:45.882578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:45.882884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:45.884756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:45.901786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 616/1238] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:45:45.909227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:46.089274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fee03-8bf3-4e0a-888c-acccd5e1f3fc None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:46.170906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:46.171829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:46.172141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:46.172506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:46.179896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:46.187826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:46.191034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:45:46.222889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:46.224932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:46.226570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 623/1239] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:46.234952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:46.419881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374856-a463-4466-aa84-f135eaf00209 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:46.474500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:46.475520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:46.475842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:46.476178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:46.477551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:46.478053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:46.480205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:46.513851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 617/1240] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Jun 16 14:45:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:46.524413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:46.693503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1ed5-ec5e-4c90-9e69-7f7328b6e226 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:46.751544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:46.752493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:46.752873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:46.753244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:46.845146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:46.854411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:46.856488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:47.021330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 624/1241] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:45:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:45:47.027674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:47.085700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:47.211621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7abb1-fa5d-4d9c-bf80-c49512386251 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:47.274987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:47.276337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:47.276678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:47.277018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:47.290594 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:47.292422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:47.317857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:47.319007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:47.320271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:47.320271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:47.320271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:47.320271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:47.320271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 618/1242] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:47.326194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:47.464552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:47.471771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebab59f-403a-4eec-9ca5-474d98d08d69 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:47.529799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:47.530780 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:47.531177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:47.531627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:47.533193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:47.533650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:47.535327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:47.554507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 625/1243] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:47.561422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:47.680066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:47.709765 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99b926-ccce-4042-987f-0cd9681c5de5 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:47.757114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:47.758070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:47.758458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:47.758870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:47.768772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:47.770858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:45:47.790463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:47.792521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:47.793713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 619/1244] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:45:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:47.802775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:47.935680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62483f0-052b-4574-a92f-5f00f6252a29 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:47.988266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4890811-e5ed-4b78-b698-3a6a498f28f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:47.989192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4890811-e5ed-4b78-b698-3a6a498f28f7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:47.989481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4890811-e5ed-4b78-b698-3a6a498f28f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:47.989767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4890811-e5ed-4b78-b698-3a6a498f28f7 None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:47.990775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4890811-e5ed-4b78-b698-3a6a498f28f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:47.991098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4890811-e5ed-4b78-b698-3a6a498f28f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:47.992659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4890811-e5ed-4b78-b698-3a6a498f28f7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:48.012224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 626/1245] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:45:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:48.018672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:48.188038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8cbe-a126-4e81-a39e-05f797c2b748 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:48.238516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3062b-8d82-4cd5-870e-243002ed818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:48.239356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3062b-8d82-4cd5-870e-243002ed818b None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:48.239744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3062b-8d82-4cd5-870e-243002ed818b None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:48.240087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3062b-8d82-4cd5-870e-243002ed818b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:48.241161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3062b-8d82-4cd5-870e-243002ed818b None admin] RBAC: Adding query filter params (role.id=5a44c123efd54f14bc8e76881f748f8b, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=848d858044ed43e69aa055491cc29d80) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:48.241476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3062b-8d82-4cd5-870e-243002ed818b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:48.243066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3062b-8d82-4cd5-870e-243002ed818b None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:48.250517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 620/1246] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:45:48 2022] GET /identity/v3/role_assignments?user.id=848d858044ed43e69aa055491cc29d80&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=5a44c123efd54f14bc8e76881f748f8b => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:50.893737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c94f47-6525-48bc-930b-efbc62fcd55e None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:50.894329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c94f47-6525-48bc-930b-efbc62fcd55e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:50.894329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c94f47-6525-48bc-930b-efbc62fcd55e None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:50.895934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 627/1247] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:45:50.910679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551909-8b21-4f1e-ab5b-056da0232c5a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:50.910679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551909-8b21-4f1e-ab5b-056da0232c5a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:50.910679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551909-8b21-4f1e-ab5b-056da0232c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:50.915950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:50.935939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0c551909-8b21-4f1e-ab5b-056da0232c5a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:45:50.937808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0c551909-8b21-4f1e-ab5b-056da0232c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:45:51.062931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 621/1248] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:45:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:45:51.095422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c94f47-6525-48bc-930b-efbc62fcd55e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:51.131461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:51.293783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c94f47-6525-48bc-930b-efbc62fcd55e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:51.352042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:51.353204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:51.353666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:51.354134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:51.375348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:45:51.390389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:51.393438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:51.395151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 628/1249] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:51.401832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551909-8b21-4f1e-ab5b-056da0232c5a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:51.572919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551909-8b21-4f1e-ab5b-056da0232c5a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:51.618697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:51.625431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:51.626215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:51.626613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:51.627095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:51.628014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:51.628328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:51.630226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:51.643233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 622/1250] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:45:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:45:51.650890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:51.764312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:51.795159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc051e3-51cb-4c37-8c57-1be3497cef78 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:51.847145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:51.848260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:51.848860 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:51.849207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:51.869001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:51.871454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:51.903796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:51.905623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:51.906634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:51.906634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:51.906634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:51.906634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:51.913124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 629/1251] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:51.916000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:52.056542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2668bf-d592-4426-a1f4-e5bda643756a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:52.105088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:52.106204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:52.107413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:52.107862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:52.109433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:52.110012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:52.112688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:52.142053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 623/1252] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:45:52.153870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:52.316057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe21a5-5c36-41a1-9c57-8bd242cf3f49 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:52.357635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:52.374803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:52.375253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:52.375783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:52.376258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:52.391159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:52.393946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:45:52.426054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:52.428222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:52.430215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 630/1253] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:52.437958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:52.523167 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:52.583219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851c9c5-c5dd-41ce-9a69-cd90adba3f0c None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:52.639076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:52.639879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:52.640217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:52.640673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:52.642309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:52.642834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:52.645939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:52.674028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 624/1254] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Jun 16 14:45:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:52.681499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:52.885045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20351c7-8cbc-46d4-ae90-c08d591d24e7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:52.933724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:52.936715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:52.936715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:52.936715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:53.015876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:53.034947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:53.036967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:53.201575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 631/1255] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:45:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:45:53.207784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:53.347190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbc6be-bdc4-43cb-8f06-9b1f18bb05b4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:53.363231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:53.389157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:53.390050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:53.390303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:53.390579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:53.404158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:53.405948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:53.427608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:53.428767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:53.430217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:53.430217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:53.430217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:53.430217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:53.430217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 625/1256] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:53.434922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:53.564658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:53.579707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97192cb5-8dd1-4d9f-a97f-94f57cadb63d None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:53.621278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:53.621999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:53.622856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:53.623243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:53.623883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:53.624198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:53.626091 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:53.642318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 632/1257] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:53.648673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:53.803410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214bf83-efe3-4970-9e50-96e7253da4e2 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:53.864724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:53.866057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:53.866937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:53.867054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:53.883036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:53.883931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:45:53.908005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:53.910308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:45:53.911282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 626/1258] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:53.918777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:54.063939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ca04a-f863-4f6a-836b-fd9be9ea8ff9 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:54.108666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4463159-e199-4555-b19a-48f0b554899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:54.109412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4463159-e199-4555-b19a-48f0b554899f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:54.109710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4463159-e199-4555-b19a-48f0b554899f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:54.110022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4463159-e199-4555-b19a-48f0b554899f None admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:54.110867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4463159-e199-4555-b19a-48f0b554899f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:54.111148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4463159-e199-4555-b19a-48f0b554899f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:54.112649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4463159-e199-4555-b19a-48f0b554899f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:54.126714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 633/1259] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:45:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:54.134284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:54.169730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:54.272533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56feb4a-210b-4122-bf7b-38e3c51f8d38 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:54.321897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282d781-0fba-4717-923b-de3acab706a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:54.322864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d781-0fba-4717-923b-de3acab706a1 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:54.323150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d781-0fba-4717-923b-de3acab706a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:54.323428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d781-0fba-4717-923b-de3acab706a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:54.324297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282d781-0fba-4717-923b-de3acab706a1 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=848d858044ed43e69aa055491cc29d80) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:54.324611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282d781-0fba-4717-923b-de3acab706a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:54.326192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282d781-0fba-4717-923b-de3acab706a1 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:54.333066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 627/1260] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:45:54 2022] GET /identity/v3/role_assignments?user.id=848d858044ed43e69aa055491cc29d80&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=c21668b08994436ca32efab183c5ca03 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:56.616003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909e79e-dc41-4816-b0fd-c8f12cb32886 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:56.616674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909e79e-dc41-4816-b0fd-c8f12cb32886 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:56.616674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909e79e-dc41-4816-b0fd-c8f12cb32886 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:56.618048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 634/1261] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:45:56.625206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dfb24-9e53-4f5f-a036-be43b7e8a769 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:56.625810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dfb24-9e53-4f5f-a036-be43b7e8a769 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:56.625810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dfb24-9e53-4f5f-a036-be43b7e8a769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:56.644218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8d1dfb24-9e53-4f5f-a036-be43b7e8a769 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:45:56.645761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8d1dfb24-9e53-4f5f-a036-be43b7e8a769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:45:56.721942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 628/1262] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:45:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:45:56.746118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909e79e-dc41-4816-b0fd-c8f12cb32886 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:56.759413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:56.876990 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909e79e-dc41-4816-b0fd-c8f12cb32886 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:56.918038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:56.919513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:56.919801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:56.920082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:56.932020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:45:56.940496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:56.942322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:56.943750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 635/1263] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:56.948262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dfb24-9e53-4f5f-a036-be43b7e8a769 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:56.968482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:57.063337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dfb24-9e53-4f5f-a036-be43b7e8a769 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:57.124496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:57.125696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:57.126214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:57.126731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:57.128261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:57.128747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:57.131443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:57.147743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 629/1264] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:57.156582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:57.285609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:57.348682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264b8e4-f114-406a-9aa0-0caffc93f90c None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:57.392412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:57.393721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:57.394688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:57.394688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:57.413185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:57.414744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:57.438863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:57.440263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:57.441560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:57.441560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:57.441560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:57.441560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:57.441560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 636/1265] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:57.447278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:57.564722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:57.596867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc20d0f-571b-4596-8380-eece2ec2f4bd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:57.643145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:57.643905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:57.644040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:57.644280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:57.645058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:57.645641 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:57.647076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:57.663269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 630/1266] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:57.671902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:57.790797 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b889960-b806-499f-b686-6fbf6bf7de1f None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:57.814492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:57.831899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:57.832706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:57.833032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:57.833325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:57.841529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:57.843017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:45:57.858377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:57.860085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:57.861726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 637/1267] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:57.866317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:57.974249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008008-f243-4ac0-badb-41d50ea3a2d4 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:58.011467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:58.012146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:58.012430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:58.012806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:58.014231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:58.014544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:58.016150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:58.027811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:58.038316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 631/1268] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Jun 16 14:45:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:58.045288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:58.188653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14433939-eec1-4e53-aafa-84e828186ee7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:58.238834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:58.239735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:58.240060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:58.240386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:58.255644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:58.257578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:45:58.283414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:58.285125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:58.286566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:45:58.286566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:45:58.286566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:45:58.286566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:58.286566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 638/1269] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:45:58.292218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:58.451310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe59796-3e8b-406d-ad75-262936be3d0f None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:58.494490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:58.495127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:58.495452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:58.495810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:58.496651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:58.497006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:58.499135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:58.515557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 632/1270] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:58.522628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:58.588354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:58.720126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b878980-e5b0-478a-83be-6e401cd9c348 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:58.774839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:58.775657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:58.775968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:58.776268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:58.786635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:58.788328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:45:58.806633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:45:58.808462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:45:58.810480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:45:58.813125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 639/1271] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:45:58.816085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:58.891386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:58.953116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23a78-05ed-4e89-a8c7-c8ba635942ab None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:59.011867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16b151-ef65-4d18-94e3-e001d26fab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:59.012972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b151-ef65-4d18-94e3-e001d26fab39 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:59.013391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b151-ef65-4d18-94e3-e001d26fab39 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:59.013883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16b151-ef65-4d18-94e3-e001d26fab39 None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:59.015104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16b151-ef65-4d18-94e3-e001d26fab39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:45:59.015735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16b151-ef65-4d18-94e3-e001d26fab39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:59.017943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16b151-ef65-4d18-94e3-e001d26fab39 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:59.036955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 633/1272] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:45:59.045190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:45:59.169940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:45:59.183857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a48ee-fff9-466f-b9ef-597a8b29c9d7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:45:59.243867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f8b55-5a1c-4555-9950-88371c394a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:45:59.246143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8b55-5a1c-4555-9950-88371c394a3c None admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:45:59.248342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8b55-5a1c-4555-9950-88371c394a3c None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:45:59.248821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f8b55-5a1c-4555-9950-88371c394a3c None admin] PATH_INFO: `/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/848d858044ed43e69aa055491cc29d80/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:45:59.251928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f8b55-5a1c-4555-9950-88371c394a3c None admin] RBAC: Authorizing `identity:create_grant(project_id=2f11c4ae5736434a9e903b50b7951f1a, user_id=848d858044ed43e69aa055491cc29d80, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:45:59.255745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f8b55-5a1c-4555-9950-88371c394a3c None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:45:59.275967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 640/1273] 10.222.0.72 () {58 vars in 1439 bytes} [Thu Jun 16 14:45:59 2022] PUT /identity/v3/projects/2f11c4ae5736434a9e903b50b7951f1a/users/848d858044ed43e69aa055491cc29d80/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:46:01.596533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaf72d-9095-4626-8ef2-f9a2b39dc5a0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:01.597203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaf72d-9095-4626-8ef2-f9a2b39dc5a0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:01.597203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aaf72d-9095-4626-8ef2-f9a2b39dc5a0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:01.599183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 634/1274] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:46:01.605867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9450d4-96f8-47a6-8d72-e4862ff7ad88 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:01.606869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9450d4-96f8-47a6-8d72-e4862ff7ad88 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:01.606869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9450d4-96f8-47a6-8d72-e4862ff7ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:01.629655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-4c9450d4-96f8-47a6-8d72-e4862ff7ad88 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:46:01.630856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4c9450d4-96f8-47a6-8d72-e4862ff7ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:46:01.710745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 641/1275] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:46:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 14:46:01.735821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaf72d-9095-4626-8ef2-f9a2b39dc5a0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:01.871800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aaf72d-9095-4626-8ef2-f9a2b39dc5a0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:01.892757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:01.912640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:01.913523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:01.913808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:01.914079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:01.926643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 14:46:01.934612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:01.936418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:01.937714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 635/1276] 10.222.0.72 () {56 vars in 1096 bytes} [Thu Jun 16 14:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:46:01.943246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9450d4-96f8-47a6-8d72-e4862ff7ad88 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:02.037838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:02.059386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9450d4-96f8-47a6-8d72-e4862ff7ad88 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:02.101410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:02.102180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:02.102515 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:02.102913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:02.103747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:02.104108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:02.105728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:02.116273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 642/1277] 10.222.0.72 () {56 vars in 1099 bytes} [Thu Jun 16 14:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:46:02.123285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:02.240639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0997-81b4-4ee6-82f7-75dfe97b24f8 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:02.290454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:02.291817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:02.292304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:02.292766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:02.308199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:02.310591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:46:02.333916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:02.335910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:02.337833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:46:02.337833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:46:02.337833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:46:02.337833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:02.337833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 636/1278] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:46:02.345468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:02.556392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100fe0a-e511-494e-b301-fb9c78ca39f7 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:02.625478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:02.626644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:02.627175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:02.627676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:02.629162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:02.629755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:02.632646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:02.657361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 643/1279] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:02.666174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:02.726205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:02.860013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49da90-e116-41e9-a2de-a72c276c4b10 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:02.921086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:02.922130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:02.922441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:02.922925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:02.939422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:02.942197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 14:46:02.969345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:02.971130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:02.973371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 637/1280] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:46:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:46:02.979784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:02.985882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:03.132117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782ac07-7baf-4d49-9af8-71830d9e32f4 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:03.178269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:03.178955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:03.179238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:03.179514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:03.180374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:03.180682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:03.183223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:03.206327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 644/1281] 10.222.0.72 () {56 vars in 1143 bytes} [Thu Jun 16 14:46:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:03.216410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:03.402621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:03.416272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be53cfd-e254-41f1-9576-d7e12616762a None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:03.456115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:03.457027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:03.457313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:03.457586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:03.538512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:03.540000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:03.660506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:03.673287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 638/1282] 10.222.0.72 () {58 vars in 1282 bytes} [Thu Jun 16 14:46:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 14:46:03.679626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:03.755162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:03.826989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2541-079e-41cc-8f15-0c13f3337e41 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:03.892682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:03.894084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:03.894493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:03.894783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:03.913936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:03.916212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 14:46:03.945813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:03.947633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:03.949435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 14:46:03.949435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 14:46:03.949435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 14:46:03.949435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:46:03.949435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 645/1283] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:46:03.953838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:04.074344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f8761-5515-4dae-a2e7-8f7f1d7f49ad None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:04.115063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:04.115736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:04.116016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:04.116289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:04.117044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:04.117348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:04.118715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:04.132001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 639/1284] 10.222.0.72 () {56 vars in 1109 bytes} [Thu Jun 16 14:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:04.138171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:04.259687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397aab3e-4d7b-4509-9c90-3df3ed455fa3 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:04.278062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:04.305235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:04.306206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:04.306595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:04.306974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:04.315745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:04.317316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 14:46:04.333394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:04.335016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: ERROR keystone.server.flask.application Jun 16 14:46:04.336813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 646/1285] 10.222.0.72 () {56 vars in 1111 bytes} [Thu Jun 16 14:46:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 14:46:04.343752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:04.410658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:04.468408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41b925-7967-4978-aeb0-dbe553f553dc None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:04.509412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0ba2b-73bc-414c-ad37-ed6ae95d99cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:04.510286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ba2b-73bc-414c-ad37-ed6ae95d99cc None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:04.510622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ba2b-73bc-414c-ad37-ed6ae95d99cc None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:04.510992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0ba2b-73bc-414c-ad37-ed6ae95d99cc None admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:04.511890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ba2b-73bc-414c-ad37-ed6ae95d99cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:04.512297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ba2b-73bc-414c-ad37-ed6ae95d99cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:04.514481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0ba2b-73bc-414c-ad37-ed6ae95d99cc None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:04.529331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 640/1286] 10.222.0.72 () {56 vars in 1148 bytes} [Thu Jun 16 14:46:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:04.538730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:04.657322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613011-c4a0-470e-baef-74b1eebfc6f0 None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:04.696247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278366b-fad5-44ec-90a9-4aa623c5edd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:04.696998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278366b-fad5-44ec-90a9-4aa623c5edd4 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:04.697226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278366b-fad5-44ec-90a9-4aa623c5edd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:04.697507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278366b-fad5-44ec-90a9-4aa623c5edd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:04.698365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278366b-fad5-44ec-90a9-4aa623c5edd4 None admin] RBAC: Adding query filter params (role.id=c21668b08994436ca32efab183c5ca03, scope.project.id=2f11c4ae5736434a9e903b50b7951f1a, user.id=848d858044ed43e69aa055491cc29d80) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:04.698686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278366b-fad5-44ec-90a9-4aa623c5edd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:04.700240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278366b-fad5-44ec-90a9-4aa623c5edd4 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:04.707035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 647/1287] 10.222.0.72 () {56 vars in 1374 bytes} [Thu Jun 16 14:46:04 2022] GET /identity/v3/role_assignments?user.id=848d858044ed43e69aa055491cc29d80&scope.project.id=2f11c4ae5736434a9e903b50b7951f1a&role.id=c21668b08994436ca32efab183c5ca03 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:07.032961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109197a-646a-4b6d-b159-deadf600abe2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:07.033616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109197a-646a-4b6d-b159-deadf600abe2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:07.033616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109197a-646a-4b6d-b159-deadf600abe2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:07.035056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 641/1288] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:46:07.048378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2115849-3872-481e-9b2f-9a2a231d8aba None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:07.049014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2115849-3872-481e-9b2f-9a2a231d8aba None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:07.049625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2115849-3872-481e-9b2f-9a2a231d8aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:07.080297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a2115849-3872-481e-9b2f-9a2a231d8aba None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:46:07.082432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a2115849-3872-481e-9b2f-9a2a231d8aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:46:07.134934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:07.201631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 648/1289] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:46:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:46:07.225071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109197a-646a-4b6d-b159-deadf600abe2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:07.358423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109197a-646a-4b6d-b159-deadf600abe2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:07.374605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:07.421281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:07.422170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:07.422663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:07.423171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:07.423881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:07.425837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:07.442975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:07.445502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:07.447540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 642/1290] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:46:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:46:07.451606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2115849-3872-481e-9b2f-9a2a231d8aba None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:07.591959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2115849-3872-481e-9b2f-9a2a231d8aba None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:07.632277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cea1e-6828-4ab6-becd-e305e0479c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:07.632998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cea1e-6828-4ab6-becd-e305e0479c19 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:07.633289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cea1e-6828-4ab6-becd-e305e0479c19 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:07.633569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cea1e-6828-4ab6-becd-e305e0479c19 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:07.634296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cea1e-6828-4ab6-becd-e305e0479c19 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:07.634620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cea1e-6828-4ab6-becd-e305e0479c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:07.636054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cea1e-6828-4ab6-becd-e305e0479c19 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:07.642339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 649/1291] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Jun 16 14:46:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:46:07.648546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:07.757294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05439b-c1b0-4f73-91b4-254bce05c0c0 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:07.795081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57c9a-f7f2-4e3b-acb7-ed1416024240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:07.795754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57c9a-f7f2-4e3b-acb7-ed1416024240 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:07.796042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57c9a-f7f2-4e3b-acb7-ed1416024240 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:07.796321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57c9a-f7f2-4e3b-acb7-ed1416024240 None admin] PATH_INFO: `/v3/services` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:07.797153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57c9a-f7f2-4e3b-acb7-ed1416024240 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:07.797425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57c9a-f7f2-4e3b-acb7-ed1416024240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:07.798813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57c9a-f7f2-4e3b-acb7-ed1416024240 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:07.803909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 643/1292] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Jun 16 14:46:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:10.057192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2f2b4-86e3-4760-b250-3b9a34ef0fd0 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:10.059049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2f2b4-86e3-4760-b250-3b9a34ef0fd0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:10.059049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2f2b4-86e3-4760-b250-3b9a34ef0fd0 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:10.060041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 650/1293] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:46:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:46:10.065984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace01ee8-f0fe-4eaa-bcb8-5e3cec9ff48c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:10.066547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace01ee8-f0fe-4eaa-bcb8-5e3cec9ff48c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:10.066643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace01ee8-f0fe-4eaa-bcb8-5e3cec9ff48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:10.084879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ace01ee8-f0fe-4eaa-bcb8-5e3cec9ff48c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:46:10.086169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ace01ee8-f0fe-4eaa-bcb8-5e3cec9ff48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:46:10.120772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:10.166698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 644/1294] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:46:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:46:10.196084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2f2b4-86e3-4760-b250-3b9a34ef0fd0 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:10.246476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:10.356395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2f2b4-86e3-4760-b250-3b9a34ef0fd0 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:10.396945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349b1e2-727c-4b3a-babd-086194de3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:10.397679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349b1e2-727c-4b3a-babd-086194de3041 None admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:10.398029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349b1e2-727c-4b3a-babd-086194de3041 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:10.398365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349b1e2-727c-4b3a-babd-086194de3041 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:10.399171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349b1e2-727c-4b3a-babd-086194de3041 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:10.400880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349b1e2-727c-4b3a-babd-086194de3041 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:10.416948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 651/1295] 10.222.0.72 () {60 vars in 1141 bytes} [Thu Jun 16 14:46:10 2022] POST /identity/v3/services => generated 246 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:46:12.680220 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b0eac-952f-4521-9453-fc2d30be5d03 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:12.680792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b0eac-952f-4521-9453-fc2d30be5d03 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:12.680927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b0eac-952f-4521-9453-fc2d30be5d03 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:12.682232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 645/1296] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:46:12.688105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e518d1-5030-4959-ba35-06652616a3b8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:12.688579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e518d1-5030-4959-ba35-06652616a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:12.688675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e518d1-5030-4959-ba35-06652616a3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:12.705219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-55e518d1-5030-4959-ba35-06652616a3b8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:46:12.706572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-55e518d1-5030-4959-ba35-06652616a3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:46:12.788398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 652/1297] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:46:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:46:12.813277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b0eac-952f-4521-9453-fc2d30be5d03 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:12.949468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b0eac-952f-4521-9453-fc2d30be5d03 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:12.987430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:12.988245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:12.988548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:12.988809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] PATH_INFO: `/v3/services/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:12.989496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:12.990867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:13.019498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:13.021462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:13.022857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 646/1298] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:46:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:46:13.027217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e518d1-5030-4959-ba35-06652616a3b8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:13.055409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:13.150300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e518d1-5030-4959-ba35-06652616a3b8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:13.188101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:13.188794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:13.189117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:13.189394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:13.190127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] RBAC: Adding query filter params (name=placement) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:13.190431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:13.191845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:13.200664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 653/1299] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Jun 16 14:46:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:13.206674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:13.242646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:13.329558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00e7e3-5289-452e-b234-8d116f16d5dd None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:13.365927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ad31-e813-48a0-b883-b78095057dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:13.366587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ad31-e813-48a0-b883-b78095057dfe None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:13.366868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ad31-e813-48a0-b883-b78095057dfe None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:13.367144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ad31-e813-48a0-b883-b78095057dfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:13.367979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ad31-e813-48a0-b883-b78095057dfe None admin] RBAC: Adding query filter params (interface=public, service_id=5be6a598fd3d442dba317a6b523f624d, region_id=RegionOne) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:13.368284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ad31-e813-48a0-b883-b78095057dfe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:13.370005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ad31-e813-48a0-b883-b78095057dfe None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:13.376251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 647/1300] 10.222.0.72 () {56 vars in 1251 bytes} [Thu Jun 16 14:46:13 2022] GET /identity/v3/endpoints?service_id=5be6a598fd3d442dba317a6b523f624d&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:13.382311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:13.494342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:13.529404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb486c-c7b4-4f68-80e0-008a315c045a None admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:13.596085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c246f7-0cac-4e97-b731-5b25f6c5e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:13.597281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c246f7-0cac-4e97-b731-5b25f6c5e288 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:13.597777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c246f7-0cac-4e97-b731-5b25f6c5e288 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:13.598275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c246f7-0cac-4e97-b731-5b25f6c5e288 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:13.599563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c246f7-0cac-4e97-b731-5b25f6c5e288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:13.602218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c246f7-0cac-4e97-b731-5b25f6c5e288 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:13.616988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 654/1301] 10.222.0.72 () {56 vars in 1087 bytes} [Thu Jun 16 14:46:13 2022] GET /identity/v3/services => generated 2191 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:46:16.073552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52058731-a132-42a5-ad09-83594484c1f1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:16.074177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52058731-a132-42a5-ad09-83594484c1f1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:16.074301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52058731-a132-42a5-ad09-83594484c1f1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:16.075712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 648/1302] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:46:16.086585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d7e92-3d03-436e-adba-98f2395e4106 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:16.087334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d7e92-3d03-436e-adba-98f2395e4106 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:16.087334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d7e92-3d03-436e-adba-98f2395e4106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:16.107751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7e7d7e92-3d03-436e-adba-98f2395e4106 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:46:16.109034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7e7d7e92-3d03-436e-adba-98f2395e4106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:46:16.195390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 655/1303] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:46:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 14:46:16.230700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52058731-a132-42a5-ad09-83594484c1f1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:16.393958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:16.403895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52058731-a132-42a5-ad09-83594484c1f1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:16.465769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:16.467060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:16.467570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:16.468083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:16.469446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:16.472098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 14:46:16.489061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 14:46:16.491255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application with Lock( Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self._enter() Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application return self.creator() Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application created_value = creator( Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: ERROR keystone.server.flask.application Jun 16 14:46:16.493112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 649/1304] 10.222.0.72 () {56 vars in 1117 bytes} [Thu Jun 16 14:46:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 14:46:16.499772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d7e92-3d03-436e-adba-98f2395e4106 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:16.590070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:16.620487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d7e92-3d03-436e-adba-98f2395e4106 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:16.667970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d02534-6890-4783-b9cf-a83e2155d124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:16.668687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d02534-6890-4783-b9cf-a83e2155d124 None admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:16.669005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d02534-6890-4783-b9cf-a83e2155d124 None admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:16.669286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d02534-6890-4783-b9cf-a83e2155d124 None admin] PATH_INFO: `/v3/services` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:16.670022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d02534-6890-4783-b9cf-a83e2155d124 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:46:16.670337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d02534-6890-4783-b9cf-a83e2155d124 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:16.671754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d02534-6890-4783-b9cf-a83e2155d124 None admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:16.677662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 656/1305] 10.222.0.72 () {56 vars in 1116 bytes} [Thu Jun 16 14:46:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:46:16.684388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:46:16.794747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cefc92-8d03-4be4-8922-aa01ca015c14 None admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:46:16.843539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3056f-c843-4d3e-8b30-2186830e68e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:46:16.844381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3056f-c843-4d3e-8b30-2186830e68e6 None admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:46:16.844674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3056f-c843-4d3e-8b30-2186830e68e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:46:16.845068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3056f-c843-4d3e-8b30-2186830e68e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:46:16.845980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3056f-c843-4d3e-8b30-2186830e68e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:46:16.847625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3056f-c843-4d3e-8b30-2186830e68e6 None admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:46:16.869710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 650/1306] 10.222.0.72 () {60 vars in 1144 bytes} [Thu Jun 16 14:46:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:46:26.674490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=30 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:46:26.864343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:41.361604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b985b-07a5-4760-ad8e-77f6b8d9d79e None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:41.362479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b985b-07a5-4760-ad8e-77f6b8d9d79e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:41.362479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b985b-07a5-4760-ad8e-77f6b8d9d79e None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:41.369135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 657/1307] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:47:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:47:41.371935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a9639-eb50-4acd-a16e-2863bcceb19d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:41.372503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a9639-eb50-4acd-a16e-2863bcceb19d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:41.372931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a9639-eb50-4acd-a16e-2863bcceb19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:41.666948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4d3a9639-eb50-4acd-a16e-2863bcceb19d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:47:41.669137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4d3a9639-eb50-4acd-a16e-2863bcceb19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:42.023867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 651/1308] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 655 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:47:42.137167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7d675-06d5-4372-98ce-8e1aaa5c88bb None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:42.140081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7d675-06d5-4372-98ce-8e1aaa5c88bb None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:42.141678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7d675-06d5-4372-98ce-8e1aaa5c88bb None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:42.144076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 658/1309] 10.222.0.72 () {54 vars in 964 bytes} [Thu Jun 16 14:47:42 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:47:42.159931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b782-ecd7-4e69-8edc-031e67058211 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:42.160888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b782-ecd7-4e69-8edc-031e67058211 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:42.161932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9b782-ecd7-4e69-8edc-031e67058211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:42.379317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-edd9b782-ecd7-4e69-8edc-031e67058211 None None] MFA Rules not processed for user `2fc5135327fe41409cc4ba407b004d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:47:42.383207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-edd9b782-ecd7-4e69-8edc-031e67058211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:42.456141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:42.589030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 652/1310] 10.222.0.72 () {58 vars in 1061 bytes} [Thu Jun 16 14:47:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:47:42.600579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7d675-06d5-4372-98ce-8e1aaa5c88bb None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:47:42.606059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-88b7d675-06d5-4372-98ce-8e1aaa5c88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:42.855959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7d675-06d5-4372-98ce-8e1aaa5c88bb None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:47:42.980919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503105f9-8e0c-4d7e-a3b1-8b83aca1ff6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:47:42.981603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503105f9-8e0c-4d7e-a3b1-8b83aca1ff6f service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:42.981892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503105f9-8e0c-4d7e-a3b1-8b83aca1ff6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:42.982175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503105f9-8e0c-4d7e-a3b1-8b83aca1ff6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:42.983991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-503105f9-8e0c-4d7e-a3b1-8b83aca1ff6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:43.093763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:43.183606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503105f9-8e0c-4d7e-a3b1-8b83aca1ff6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:47:43.185768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503105f9-8e0c-4d7e-a3b1-8b83aca1ff6f service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:47:43.221866 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 659/1311] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:47:46.539433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197468ee-5564-47b3-83fd-475a7214d887 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:46.540132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197468ee-5564-47b3-83fd-475a7214d887 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:46.541039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197468ee-5564-47b3-83fd-475a7214d887 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:46.543838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 653/1312] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:47:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:47:46.551300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da52f4-59ff-43f5-81b1-15747a906298 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:46.552172 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da52f4-59ff-43f5-81b1-15747a906298 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:46.552485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da52f4-59ff-43f5-81b1-15747a906298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:46.663607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c1da52f4-59ff-43f5-81b1-15747a906298 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:47:46.666533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c1da52f4-59ff-43f5-81b1-15747a906298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:46.886211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 660/1313] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:47:46.953375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8f92e-4e3a-4d79-99da-2415c3824818 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:46.953806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8f92e-4e3a-4d79-99da-2415c3824818 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:46.954161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8f92e-4e3a-4d79-99da-2415c3824818 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:46.955721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 654/1314] 10.222.0.72 () {54 vars in 964 bytes} [Thu Jun 16 14:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:47:46.964352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c71f-2b43-4dc4-91f1-15bf32cd51d1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:46.964871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c71f-2b43-4dc4-91f1-15bf32cd51d1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:46.965348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c71f-2b43-4dc4-91f1-15bf32cd51d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:46.985053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:47.090423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5dd4c71f-2b43-4dc4-91f1-15bf32cd51d1 None None] MFA Rules not processed for user `2fc5135327fe41409cc4ba407b004d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:47:47.092501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5dd4c71f-2b43-4dc4-91f1-15bf32cd51d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:47.206010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 661/1315] 10.222.0.72 () {58 vars in 1061 bytes} [Thu Jun 16 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:47:47.221122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8f92e-4e3a-4d79-99da-2415c3824818 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:47:47.223720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-32a8f92e-4e3a-4d79-99da-2415c3824818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:47.407053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8f92e-4e3a-4d79-99da-2415c3824818 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:47:47.525237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:47.531233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3288-c0e6-4271-b97d-3e2651638146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:47:47.532345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3288-c0e6-4271-b97d-3e2651638146 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:47.532821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3288-c0e6-4271-b97d-3e2651638146 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:47.533328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3288-c0e6-4271-b97d-3e2651638146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:47.535416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4b1f3288-c0e6-4271-b97d-3e2651638146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:47.694694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f3288-c0e6-4271-b97d-3e2651638146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:47:47.696467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f3288-c0e6-4271-b97d-3e2651638146 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:47:47.720245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 655/1316] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:47:51.299937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367eb7-e79f-4e6b-8b16-793cff16658d None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:51.300575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367eb7-e79f-4e6b-8b16-793cff16658d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:51.300575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367eb7-e79f-4e6b-8b16-793cff16658d None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:51.302131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 662/1317] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:47:51.310013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013f86-a69f-4b9b-b6b9-62d70fc851d6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:51.310648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013f86-a69f-4b9b-b6b9-62d70fc851d6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:51.310941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013f86-a69f-4b9b-b6b9-62d70fc851d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:51.411639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-72013f86-a69f-4b9b-b6b9-62d70fc851d6 None None] MFA Rules not processed for user `c2befa26411b4fe5943e212d37990575`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:47:51.413166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-72013f86-a69f-4b9b-b6b9-62d70fc851d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:51.518646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:51.593795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 656/1318] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:47:51 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:47:51.637473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367eb7-e79f-4e6b-8b16-793cff16658d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:47:51.640453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5c367eb7-e79f-4e6b-8b16-793cff16658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:51.827133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367eb7-e79f-4e6b-8b16-793cff16658d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:47:51.833032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:51.952281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73d092-b8cf-458d-9f46-390a125cffa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:47:51.953236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73d092-b8cf-458d-9f46-390a125cffa3 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:51.953631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73d092-b8cf-458d-9f46-390a125cffa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:51.953929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73d092-b8cf-458d-9f46-390a125cffa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:51.956044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fe73d092-b8cf-458d-9f46-390a125cffa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:52.123221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73d092-b8cf-458d-9f46-390a125cffa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:47:52.124957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73d092-b8cf-458d-9f46-390a125cffa3 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:47:52.150251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 663/1319] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:47:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:47:55.594379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a4800-13e9-4c2f-ae87-f423d4084951 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:55.595016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a4800-13e9-4c2f-ae87-f423d4084951 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:55.595165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a4800-13e9-4c2f-ae87-f423d4084951 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:55.596690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 657/1320] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:47:55.604556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b41e4-467a-4e10-8603-81683a110c06 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:55.605236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b41e4-467a-4e10-8603-81683a110c06 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:55.605557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b41e4-467a-4e10-8603-81683a110c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:55.675117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:55.712026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fe9b41e4-467a-4e10-8603-81683a110c06 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:47:55.713442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fe9b41e4-467a-4e10-8603-81683a110c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:55.861978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 664/1321] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:47:55.904435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a4800-13e9-4c2f-ae87-f423d4084951 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:47:55.906230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-131a4800-13e9-4c2f-ae87-f423d4084951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:56.047449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a4800-13e9-4c2f-ae87-f423d4084951 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:47:56.145087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f5b4-e77e-42a7-89ab-6eba9c2403f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:47:56.145802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f5b4-e77e-42a7-89ab-6eba9c2403f3 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:47:56.146438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f5b4-e77e-42a7-89ab-6eba9c2403f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:47:56.146808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f5b4-e77e-42a7-89ab-6eba9c2403f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:47:56.148777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-50c1f5b4-e77e-42a7-89ab-6eba9c2403f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:47:56.158536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:47:56.312264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1f5b4-e77e-42a7-89ab-6eba9c2403f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:47:56.315447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1f5b4-e77e-42a7-89ab-6eba9c2403f3 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:47:56.352987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 658/1322] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:47:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:01.374105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe43c-31d5-41af-8db3-8cf568e64cb6 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:01.375028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe43c-31d5-41af-8db3-8cf568e64cb6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:01.375028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe43c-31d5-41af-8db3-8cf568e64cb6 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:01.376567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 665/1323] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:48:01.383701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1238-2c3d-4fe0-8e2b-e639146bc88c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:01.384386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1238-2c3d-4fe0-8e2b-e639146bc88c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:01.384725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1238-2c3d-4fe0-8e2b-e639146bc88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:01.529296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-dbea1238-2c3d-4fe0-8e2b-e639146bc88c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:01.531608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dbea1238-2c3d-4fe0-8e2b-e639146bc88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:01.658804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:01.750215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 659/1324] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:01.815382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfe43c-31d5-41af-8db3-8cf568e64cb6 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:01.818859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a6dfe43c-31d5-41af-8db3-8cf568e64cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:02.048567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfe43c-31d5-41af-8db3-8cf568e64cb6 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:02.075214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:02.192694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb390ff-001d-463f-868d-96f7265a2e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:02.193673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb390ff-001d-463f-868d-96f7265a2e6a service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:02.194092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb390ff-001d-463f-868d-96f7265a2e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:02.194503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb390ff-001d-463f-868d-96f7265a2e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:02.199840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7fb390ff-001d-463f-868d-96f7265a2e6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:02.365893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb390ff-001d-463f-868d-96f7265a2e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:02.367409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb390ff-001d-463f-868d-96f7265a2e6a service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:02.392314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 666/1325] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:05.696475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432e40c-ef37-47ad-9de3-6dc472371d98 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:05.697182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432e40c-ef37-47ad-9de3-6dc472371d98 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:05.697182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432e40c-ef37-47ad-9de3-6dc472371d98 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:05.698451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 660/1326] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:05.707721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb3105-bf40-4215-9593-871df3d61d85 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:05.708361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb3105-bf40-4215-9593-871df3d61d85 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:05.708717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb3105-bf40-4215-9593-871df3d61d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:05.813368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:05.840374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-38fb3105-bf40-4215-9593-871df3d61d85 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:05.842292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38fb3105-bf40-4215-9593-871df3d61d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:06.061218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 667/1327] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:06.123260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432e40c-ef37-47ad-9de3-6dc472371d98 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:06.125065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7432e40c-ef37-47ad-9de3-6dc472371d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:06.288539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432e40c-ef37-47ad-9de3-6dc472371d98 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:06.427294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02568931-9feb-427b-8bf3-f30f4fe748cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:06.428358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02568931-9feb-427b-8bf3-f30f4fe748cc service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:06.428883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02568931-9feb-427b-8bf3-f30f4fe748cc service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:06.429363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02568931-9feb-427b-8bf3-f30f4fe748cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:06.433255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-02568931-9feb-427b-8bf3-f30f4fe748cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:06.469642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:06.624143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02568931-9feb-427b-8bf3-f30f4fe748cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:06.626854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02568931-9feb-427b-8bf3-f30f4fe748cc service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:06.656562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 661/1328] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:09.300514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da27e3b-bedf-4421-b9c4-f12a07948c57 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:09.301059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da27e3b-bedf-4421-b9c4-f12a07948c57 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:09.301556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da27e3b-bedf-4421-b9c4-f12a07948c57 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:09.304219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 668/1329] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:09.313058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423df180-55a6-4f3d-b617-d1e7c873a9fa None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:09.314617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423df180-55a6-4f3d-b617-d1e7c873a9fa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:09.316152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423df180-55a6-4f3d-b617-d1e7c873a9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:09.449525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-423df180-55a6-4f3d-b617-d1e7c873a9fa None None] MFA Rules not processed for user `c2befa26411b4fe5943e212d37990575`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:09.450945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-423df180-55a6-4f3d-b617-d1e7c873a9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:09.553121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:09.598294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 662/1330] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:09 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:09.647730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da27e3b-bedf-4421-b9c4-f12a07948c57 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:09.649493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-9da27e3b-bedf-4421-b9c4-f12a07948c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:09.797835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da27e3b-bedf-4421-b9c4-f12a07948c57 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:09.828136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:09.904813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1d0ad-06a5-4435-aca1-e17d6eabc763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:09.906317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d0ad-06a5-4435-aca1-e17d6eabc763 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:09.906779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d0ad-06a5-4435-aca1-e17d6eabc763 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:09.907219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1d0ad-06a5-4435-aca1-e17d6eabc763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:09.910969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-9da1d0ad-06a5-4435-aca1-e17d6eabc763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:10.124446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1d0ad-06a5-4435-aca1-e17d6eabc763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:10.125514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1d0ad-06a5-4435-aca1-e17d6eabc763 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:10.159056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 669/1331] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:09 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:13.032681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb584cc-8fd2-41f7-8334-6fca32fb0a94 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:13.033544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb584cc-8fd2-41f7-8334-6fca32fb0a94 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:13.033544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb584cc-8fd2-41f7-8334-6fca32fb0a94 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:13.034946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 663/1332] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:13.041721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce02db0-dbc1-452e-948c-e904553359d3 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:13.042581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce02db0-dbc1-452e-948c-e904553359d3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:13.042773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce02db0-dbc1-452e-948c-e904553359d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:13.137499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:13.142916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8ce02db0-dbc1-452e-948c-e904553359d3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:13.144778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8ce02db0-dbc1-452e-948c-e904553359d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:13.291700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 670/1333] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:13.329824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb584cc-8fd2-41f7-8334-6fca32fb0a94 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:13.332796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4eb584cc-8fd2-41f7-8334-6fca32fb0a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:13.457150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb584cc-8fd2-41f7-8334-6fca32fb0a94 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:13.461803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4ec1f-e478-4a21-984d-fdc18c5031e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:13.462391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4ec1f-e478-4a21-984d-fdc18c5031e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:13.462678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4ec1f-e478-4a21-984d-fdc18c5031e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:13.462955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4ec1f-e478-4a21-984d-fdc18c5031e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:13.474445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4ec1f-e478-4a21-984d-fdc18c5031e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:13.477956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4ec1f-e478-4a21-984d-fdc18c5031e6 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:13.496467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 664/1334] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:15.892479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d040-7f3d-4775-9b82-eaa884e13b34 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:15.893139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d040-7f3d-4775-9b82-eaa884e13b34 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:15.893318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d040-7f3d-4775-9b82-eaa884e13b34 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:15.895673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 671/1335] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:15.904176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42b0c-4b48-468c-be91-67a859f93598 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:15.905077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42b0c-4b48-468c-be91-67a859f93598 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:15.905077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42b0c-4b48-468c-be91-67a859f93598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:15.928684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-15f42b0c-4b48-468c-be91-67a859f93598 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:15.930823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-15f42b0c-4b48-468c-be91-67a859f93598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:15.943578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 665/1336] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:48:15.995390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304d040-7f3d-4775-9b82-eaa884e13b34 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:15.997565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0304d040-7f3d-4775-9b82-eaa884e13b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:16.024660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304d040-7f3d-4775-9b82-eaa884e13b34 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:16.030274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14df2c3-ca82-4676-93b1-97246160df52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:16.030942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df2c3-ca82-4676-93b1-97246160df52 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:16.031215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df2c3-ca82-4676-93b1-97246160df52 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:16.031490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df2c3-ca82-4676-93b1-97246160df52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:16.043318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14df2c3-ca82-4676-93b1-97246160df52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:16.044817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14df2c3-ca82-4676-93b1-97246160df52 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:16.057002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 672/1337] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:48:19.415803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6af6f3-03e6-433b-bd76-5c281fe41de0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:19.416916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6af6f3-03e6-433b-bd76-5c281fe41de0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:19.416916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6af6f3-03e6-433b-bd76-5c281fe41de0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:19.418350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 666/1338] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:19.426010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecd09-97d1-4695-b667-61ad2c4406ec None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:19.426943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecd09-97d1-4695-b667-61ad2c4406ec None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:19.426943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecd09-97d1-4695-b667-61ad2c4406ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:19.460423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-634ecd09-97d1-4695-b667-61ad2c4406ec None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:19.462433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-634ecd09-97d1-4695-b667-61ad2c4406ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:19.563772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 673/1339] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:19.601976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6af6f3-03e6-433b-bd76-5c281fe41de0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:19.614901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6af6f3-03e6-433b-bd76-5c281fe41de0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:19.619429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86ba27-5dc2-4a5d-9f44-956e5aa3359a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:19.620119 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ba27-5dc2-4a5d-9f44-956e5aa3359a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:19.620399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ba27-5dc2-4a5d-9f44-956e5aa3359a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:19.620677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ba27-5dc2-4a5d-9f44-956e5aa3359a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:19.631488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86ba27-5dc2-4a5d-9f44-956e5aa3359a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:19.633697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86ba27-5dc2-4a5d-9f44-956e5aa3359a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:19.645200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 667/1340] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:25.121722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6c578-f2d9-49e9-abc6-f5d8d7f8d45b None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:25.122637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6c578-f2d9-49e9-abc6-f5d8d7f8d45b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:25.123082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6c578-f2d9-49e9-abc6-f5d8d7f8d45b None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:25.125472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 674/1341] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:25.131390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd866a1c-fc0d-4e02-844a-04e83ba92fb0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:25.132588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd866a1c-fc0d-4e02-844a-04e83ba92fb0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:25.133573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd866a1c-fc0d-4e02-844a-04e83ba92fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:25.169052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-cd866a1c-fc0d-4e02-844a-04e83ba92fb0 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:25.171740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cd866a1c-fc0d-4e02-844a-04e83ba92fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:25.191811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 668/1342] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:25.236086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6c578-f2d9-49e9-abc6-f5d8d7f8d45b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:25.255057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6c578-f2d9-49e9-abc6-f5d8d7f8d45b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:25.262627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f473d9-06ce-44e6-83d8-a8b8d44db11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:25.264040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f473d9-06ce-44e6-83d8-a8b8d44db11e service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:25.264751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f473d9-06ce-44e6-83d8-a8b8d44db11e service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:25.265263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f473d9-06ce-44e6-83d8-a8b8d44db11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:25.282236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f473d9-06ce-44e6-83d8-a8b8d44db11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:25.285716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f473d9-06ce-44e6-83d8-a8b8d44db11e service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:25.305918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 675/1343] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:48:28.625456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce2c8-01ba-47d5-9c3a-b2def05a14fd None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:28.625456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce2c8-01ba-47d5-9c3a-b2def05a14fd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:28.625456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce2c8-01ba-47d5-9c3a-b2def05a14fd None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:28.625456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 669/1344] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:28.632034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2226-b1ea-47be-9c43-8d75a12bf2b2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:28.632699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2226-b1ea-47be-9c43-8d75a12bf2b2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:28.634030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e2226-b1ea-47be-9c43-8d75a12bf2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:28.660095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8b9e2226-b1ea-47be-9c43-8d75a12bf2b2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:28.661364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8b9e2226-b1ea-47be-9c43-8d75a12bf2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:28.672739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 676/1345] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:48:28.739609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce2c8-01ba-47d5-9c3a-b2def05a14fd None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:28.755936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce2c8-01ba-47d5-9c3a-b2def05a14fd None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:28.759896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80ef44-03f1-4b8b-bd3a-ad2267544ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:28.760572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80ef44-03f1-4b8b-bd3a-ad2267544ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:28.760870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80ef44-03f1-4b8b-bd3a-ad2267544ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:28.761177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80ef44-03f1-4b8b-bd3a-ad2267544ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:28.774926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80ef44-03f1-4b8b-bd3a-ad2267544ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:28.777764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b80ef44-03f1-4b8b-bd3a-ad2267544ab5 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:28.792118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 670/1346] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:33.929201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e5782-69cd-445c-8025-48689dbd5701 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:33.929837 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e5782-69cd-445c-8025-48689dbd5701 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:33.929837 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e5782-69cd-445c-8025-48689dbd5701 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:33.931199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 677/1347] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:33.937042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6740f4-90af-4050-8f2b-18ee5c6fa77b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:33.937685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6740f4-90af-4050-8f2b-18ee5c6fa77b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:33.938255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6740f4-90af-4050-8f2b-18ee5c6fa77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:33.963258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2e6740f4-90af-4050-8f2b-18ee5c6fa77b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:33.965243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2e6740f4-90af-4050-8f2b-18ee5c6fa77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:33.974138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 671/1348] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:34.015172 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e5782-69cd-445c-8025-48689dbd5701 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:34.035610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e5782-69cd-445c-8025-48689dbd5701 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:34.042455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004dae3-0e25-4241-83dd-9bf374359bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:34.043609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004dae3-0e25-4241-83dd-9bf374359bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:34.044097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004dae3-0e25-4241-83dd-9bf374359bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:34.044588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004dae3-0e25-4241-83dd-9bf374359bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:34.055186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:34.063197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004dae3-0e25-4241-83dd-9bf374359bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:34.066090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004dae3-0e25-4241-83dd-9bf374359bb7 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:34.086217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 678/1349] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:40.693596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76304df2-2ccd-475e-b76d-51e612db3f08 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:40.693596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76304df2-2ccd-475e-b76d-51e612db3f08 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:40.693596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76304df2-2ccd-475e-b76d-51e612db3f08 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:40.695889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 672/1350] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 14:48:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:40.707115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0326571-f4bb-43bf-97aa-816c774dc02d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:40.707698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0326571-f4bb-43bf-97aa-816c774dc02d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:40.708321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0326571-f4bb-43bf-97aa-816c774dc02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:40.879855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a0326571-f4bb-43bf-97aa-816c774dc02d None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:40.883138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a0326571-f4bb-43bf-97aa-816c774dc02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:41.084874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 679/1351] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 14:48:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:43.955628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=75 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:48.214011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847deb77-6406-472d-b080-3d0ff74119bd None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:48.215050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847deb77-6406-472d-b080-3d0ff74119bd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:48.215326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847deb77-6406-472d-b080-3d0ff74119bd None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:48.217011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 673/1352] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 14:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:48.226659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761e94-22e5-4970-a300-0ccb1c97916a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:48.227213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761e94-22e5-4970-a300-0ccb1c97916a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:48.227894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a761e94-22e5-4970-a300-0ccb1c97916a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:48.255384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2a761e94-22e5-4970-a300-0ccb1c97916a None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:48.256732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2a761e94-22e5-4970-a300-0ccb1c97916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:48.267188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 680/1353] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:50.031036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef8de0-af77-4b0c-b519-1755a3de86c6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:50.031882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef8de0-af77-4b0c-b519-1755a3de86c6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:50.032167 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef8de0-af77-4b0c-b519-1755a3de86c6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:50.034900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 674/1354] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 14:48:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:50.043698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af793f8-18aa-4d2a-be22-3177790356d6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:50.044254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af793f8-18aa-4d2a-be22-3177790356d6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:50.044557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af793f8-18aa-4d2a-be22-3177790356d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:50.068151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2af793f8-18aa-4d2a-be22-3177790356d6 None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:50.069990 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2af793f8-18aa-4d2a-be22-3177790356d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:50.086716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 681/1355] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 14:48:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:54.891192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781ee59-cd32-4c99-996e-b3a8c4b76646 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:54.891894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781ee59-cd32-4c99-996e-b3a8c4b76646 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:54.892100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781ee59-cd32-4c99-996e-b3a8c4b76646 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:54.898235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 675/1356] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:54.905856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b86aa5-74bd-407e-815c-a45f7484efb2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:54.907540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b86aa5-74bd-407e-815c-a45f7484efb2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:54.908500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b86aa5-74bd-407e-815c-a45f7484efb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:54.941150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-93b86aa5-74bd-407e-815c-a45f7484efb2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:54.942817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-93b86aa5-74bd-407e-815c-a45f7484efb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:54.954667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 682/1357] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:55.021893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b960615-684b-4708-af80-73021a73f4de None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:55.024340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b960615-684b-4708-af80-73021a73f4de None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:55.025528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b960615-684b-4708-af80-73021a73f4de None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:55.029265 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 676/1358] 10.222.0.72 () {54 vars in 961 bytes} [Thu Jun 16 14:48:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:48:55.040230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defbf8b-22e3-49d1-9976-9a7c8137dd35 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:55.040914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defbf8b-22e3-49d1-9976-9a7c8137dd35 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:55.041394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defbf8b-22e3-49d1-9976-9a7c8137dd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:55.106237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8defbf8b-22e3-49d1-9976-9a7c8137dd35 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:55.107660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8defbf8b-22e3-49d1-9976-9a7c8137dd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:55.204607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:48:55.281729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 683/1359] 10.222.0.72 () {58 vars in 1058 bytes} [Thu Jun 16 14:48:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:55.295809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b960615-684b-4708-af80-73021a73f4de None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:55.327653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b960615-684b-4708-af80-73021a73f4de None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:55.349874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499d5b-db7b-4b3a-8f4d-051b1f5610e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:55.351143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499d5b-db7b-4b3a-8f4d-051b1f5610e4 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:55.351654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499d5b-db7b-4b3a-8f4d-051b1f5610e4 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:55.352167 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499d5b-db7b-4b3a-8f4d-051b1f5610e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:55.369933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499d5b-db7b-4b3a-8f4d-051b1f5610e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:55.373218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499d5b-db7b-4b3a-8f4d-051b1f5610e4 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:55.394877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 677/1360] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:48:58.957186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99756bf6-4b36-44b9-9c80-1d76d4e3d1ce None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:58.957842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99756bf6-4b36-44b9-9c80-1d76d4e3d1ce None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:58.958124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99756bf6-4b36-44b9-9c80-1d76d4e3d1ce None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:58.960654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 684/1361] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:48:58.969705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463a1b7-f804-4bdb-b178-7249a7dd3edf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:58.971210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463a1b7-f804-4bdb-b178-7249a7dd3edf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:58.971677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463a1b7-f804-4bdb-b178-7249a7dd3edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:59.013642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4463a1b7-f804-4bdb-b178-7249a7dd3edf None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:48:59.016180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4463a1b7-f804-4bdb-b178-7249a7dd3edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:48:59.032084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 678/1362] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:48:59.107932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99756bf6-4b36-44b9-9c80-1d76d4e3d1ce None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:48:59.124937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99756bf6-4b36-44b9-9c80-1d76d4e3d1ce None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:48:59.132433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225478c1-f1e2-4dfa-aaff-4873c3796a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:48:59.133424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225478c1-f1e2-4dfa-aaff-4873c3796a9a service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:48:59.133885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225478c1-f1e2-4dfa-aaff-4873c3796a9a service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:48:59.134335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225478c1-f1e2-4dfa-aaff-4873c3796a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:48:59.146679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225478c1-f1e2-4dfa-aaff-4873c3796a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:48:59.149932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225478c1-f1e2-4dfa-aaff-4873c3796a9a service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:48:59.166620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 685/1363] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:49:02.407491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510d2f9-1700-4657-a673-b177b5ede6a0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:02.408440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510d2f9-1700-4657-a673-b177b5ede6a0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:02.408440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3510d2f9-1700-4657-a673-b177b5ede6a0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:02.410562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 679/1364] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:02.417091 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e17ef-eeff-4bf7-b37c-1620c3ad9b06 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:02.417676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e17ef-eeff-4bf7-b37c-1620c3ad9b06 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:02.417676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e17ef-eeff-4bf7-b37c-1620c3ad9b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:02.443658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-378e17ef-eeff-4bf7-b37c-1620c3ad9b06 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:02.444972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-378e17ef-eeff-4bf7-b37c-1620c3ad9b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:02.459554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 686/1365] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:02.519574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4cd33-a21c-4d9a-bebb-f5c92d54b46c None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:02.519981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4cd33-a21c-4d9a-bebb-f5c92d54b46c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:02.520469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4cd33-a21c-4d9a-bebb-f5c92d54b46c None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:02.523165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 680/1366] 10.222.0.72 () {54 vars in 961 bytes} [Thu Jun 16 14:49:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:02.530282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e7145-d7a4-48e5-a6e0-62e3ebe9ca24 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:02.530562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e7145-d7a4-48e5-a6e0-62e3ebe9ca24 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:02.530842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e7145-d7a4-48e5-a6e0-62e3ebe9ca24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:02.550645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ce8e7145-d7a4-48e5-a6e0-62e3ebe9ca24 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:02.551885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ce8e7145-d7a4-48e5-a6e0-62e3ebe9ca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:02.562678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 687/1367] 10.222.0.72 () {58 vars in 1058 bytes} [Thu Jun 16 14:49:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:02.572448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4cd33-a21c-4d9a-bebb-f5c92d54b46c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:02.601794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4cd33-a21c-4d9a-bebb-f5c92d54b46c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:02.613483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72d675-c531-459f-9aea-ab67e15fed97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:02.614618 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72d675-c531-459f-9aea-ab67e15fed97 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:02.615135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72d675-c531-459f-9aea-ab67e15fed97 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:02.615652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72d675-c531-459f-9aea-ab67e15fed97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:02.633936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72d675-c531-459f-9aea-ab67e15fed97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:02.637132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72d675-c531-459f-9aea-ab67e15fed97 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:02.658199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 681/1368] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:49:05.245920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84f042-5861-408b-b7b1-4045b1c2dadc None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:05.246482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84f042-5861-408b-b7b1-4045b1c2dadc None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:05.246482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84f042-5861-408b-b7b1-4045b1c2dadc None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:05.248409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 688/1369] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:05.254671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed1e65-958f-4938-b808-17f2e672f760 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:05.255212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed1e65-958f-4938-b808-17f2e672f760 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:05.255292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed1e65-958f-4938-b808-17f2e672f760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:05.276871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-73ed1e65-958f-4938-b808-17f2e672f760 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:05.278357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-73ed1e65-958f-4938-b808-17f2e672f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:05.293044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 682/1370] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:05.345920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84f042-5861-408b-b7b1-4045b1c2dadc None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:05.374890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84f042-5861-408b-b7b1-4045b1c2dadc None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:05.384600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f408-cd6b-4830-bbbb-296454bfd996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:05.385757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f408-cd6b-4830-bbbb-296454bfd996 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:05.386343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f408-cd6b-4830-bbbb-296454bfd996 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:05.386784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f408-cd6b-4830-bbbb-296454bfd996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:05.403427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58f408-cd6b-4830-bbbb-296454bfd996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:05.405772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58f408-cd6b-4830-bbbb-296454bfd996 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:05.424827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 689/1371] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:49:06.401380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742df25-ad17-46d1-9857-8dd80efccf61 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:06.402261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742df25-ad17-46d1-9857-8dd80efccf61 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:06.402261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742df25-ad17-46d1-9857-8dd80efccf61 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:06.404245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 683/1372] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:06.410691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d108f9-ccf8-4ba9-b2d3-f3f64d7fce3e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:06.411493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d108f9-ccf8-4ba9-b2d3-f3f64d7fce3e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:06.411493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d108f9-ccf8-4ba9-b2d3-f3f64d7fce3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:06.433301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-48d108f9-ccf8-4ba9-b2d3-f3f64d7fce3e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:06.434534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-48d108f9-ccf8-4ba9-b2d3-f3f64d7fce3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:06.443484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 690/1373] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:49:06.762606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cba1e-c18e-4c09-834e-350e39dd53f3 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:06.762606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cba1e-c18e-4c09-834e-350e39dd53f3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:06.762606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cba1e-c18e-4c09-834e-350e39dd53f3 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:06.766573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 684/1374] 10.222.0.72 () {54 vars in 963 bytes} [Thu Jun 16 14:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:06.770163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d1eab-526e-4f04-a3f9-0c6ae69821c0 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:06.770163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d1eab-526e-4f04-a3f9-0c6ae69821c0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:06.770163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d1eab-526e-4f04-a3f9-0c6ae69821c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:06.848934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-154d1eab-526e-4f04-a3f9-0c6ae69821c0 None None] MFA Rules not processed for user `6812dddc3db746b6893f12f1ae0c715d`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:06.851052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-154d1eab-526e-4f04-a3f9-0c6ae69821c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:06.950823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 691/1375] 10.222.0.72 () {58 vars in 1060 bytes} [Thu Jun 16 14:49:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:49:06.961904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cba1e-c18e-4c09-834e-350e39dd53f3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:06.981985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cba1e-c18e-4c09-834e-350e39dd53f3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:06.986074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e78c9e-fb66-4c1e-8ed5-d51d9fc4af22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:06.986849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e78c9e-fb66-4c1e-8ed5-d51d9fc4af22 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:06.987186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e78c9e-fb66-4c1e-8ed5-d51d9fc4af22 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:06.987715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e78c9e-fb66-4c1e-8ed5-d51d9fc4af22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:06.999339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e78c9e-fb66-4c1e-8ed5-d51d9fc4af22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:07.000263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e78c9e-fb66-4c1e-8ed5-d51d9fc4af22 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:07.014728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 685/1376] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:49:08.275547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6285a-aac6-4553-a147-7640ad715b09 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:08.276343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6285a-aac6-4553-a147-7640ad715b09 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:08.276343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6285a-aac6-4553-a147-7640ad715b09 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:08.277520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 692/1377] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:08.285059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564da1-1ba7-44f9-8195-d2d91e0b4869 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:08.285668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564da1-1ba7-44f9-8195-d2d91e0b4869 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:08.285971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f564da1-1ba7-44f9-8195-d2d91e0b4869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:08.310639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3f564da1-1ba7-44f9-8195-d2d91e0b4869 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:08.312045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3f564da1-1ba7-44f9-8195-d2d91e0b4869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:08.325941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 686/1378] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:08.382430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6285a-aac6-4553-a147-7640ad715b09 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:08.399276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6285a-aac6-4553-a147-7640ad715b09 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:08.409821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58645455-13a1-4477-8834-89d38d88a191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:08.410826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58645455-13a1-4477-8834-89d38d88a191 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:08.411138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58645455-13a1-4477-8834-89d38d88a191 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:08.411420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58645455-13a1-4477-8834-89d38d88a191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:08.424311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58645455-13a1-4477-8834-89d38d88a191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:08.427259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58645455-13a1-4477-8834-89d38d88a191 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:08.447380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 693/1379] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:49:10.937169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8cf34-f164-4e06-b47d-bbf991ab1778 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:10.937903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8cf34-f164-4e06-b47d-bbf991ab1778 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:10.937903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8cf34-f164-4e06-b47d-bbf991ab1778 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:10.939365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 687/1380] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:10.947715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0dbdc-679f-4148-8f1d-d56c75fea78f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:10.948464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0dbdc-679f-4148-8f1d-d56c75fea78f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:10.948464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0dbdc-679f-4148-8f1d-d56c75fea78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:10.968828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-63d0dbdc-679f-4148-8f1d-d56c75fea78f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:10.971781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-63d0dbdc-679f-4148-8f1d-d56c75fea78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:10.989489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 694/1381] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:11.060287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8cf34-f164-4e06-b47d-bbf991ab1778 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:11.076671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8cf34-f164-4e06-b47d-bbf991ab1778 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:11.082568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805fe42-d7a8-4ad3-bfdc-465a9cf6a88f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:11.083310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805fe42-d7a8-4ad3-bfdc-465a9cf6a88f service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:11.083535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805fe42-d7a8-4ad3-bfdc-465a9cf6a88f service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:11.083817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805fe42-d7a8-4ad3-bfdc-465a9cf6a88f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:11.093869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805fe42-d7a8-4ad3-bfdc-465a9cf6a88f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:11.095505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805fe42-d7a8-4ad3-bfdc-465a9cf6a88f service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:11.107156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 688/1382] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:49:13.488888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55bc2c-2c73-411b-add9-122ba411de7a None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:13.489726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55bc2c-2c73-411b-add9-122ba411de7a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:13.489726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55bc2c-2c73-411b-add9-122ba411de7a None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:13.491806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 695/1383] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:13.497592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3428d-b19d-402c-b358-ed0e55e25d96 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:13.498585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3428d-b19d-402c-b358-ed0e55e25d96 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:13.499101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3428d-b19d-402c-b358-ed0e55e25d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:13.519575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e4e3428d-b19d-402c-b358-ed0e55e25d96 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:13.521256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e4e3428d-b19d-402c-b358-ed0e55e25d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:13.530660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 689/1384] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:49:13.576445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55bc2c-2c73-411b-add9-122ba411de7a None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:13.603524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55bc2c-2c73-411b-add9-122ba411de7a None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:13.615294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6bcec-2e8d-451c-b776-76aa87501369 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:13.617315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6bcec-2e8d-451c-b776-76aa87501369 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:13.617955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6bcec-2e8d-451c-b776-76aa87501369 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:13.618572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6bcec-2e8d-451c-b776-76aa87501369 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:13.634967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6bcec-2e8d-451c-b776-76aa87501369 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:13.638173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6bcec-2e8d-451c-b776-76aa87501369 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:13.657978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 696/1385] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:49:16.408499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68869b-fb23-4fd4-8f5e-43fc938cd306 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:16.409355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68869b-fb23-4fd4-8f5e-43fc938cd306 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:16.409355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68869b-fb23-4fd4-8f5e-43fc938cd306 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:16.411060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 690/1386] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:16.419300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e972ef-f07f-4f9b-a090-bedc60b3addf None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:16.419836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e972ef-f07f-4f9b-a090-bedc60b3addf None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:16.420275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e972ef-f07f-4f9b-a090-bedc60b3addf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:16.443401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-33e972ef-f07f-4f9b-a090-bedc60b3addf None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:16.444804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-33e972ef-f07f-4f9b-a090-bedc60b3addf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:16.453351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 697/1387] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:16.504410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68869b-fb23-4fd4-8f5e-43fc938cd306 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:16.520655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68869b-fb23-4fd4-8f5e-43fc938cd306 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:16.530864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909541cc-df49-4cc7-a0e4-30b4c5f5fd33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:16.532320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909541cc-df49-4cc7-a0e4-30b4c5f5fd33 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:16.533069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909541cc-df49-4cc7-a0e4-30b4c5f5fd33 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:16.533786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909541cc-df49-4cc7-a0e4-30b4c5f5fd33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:16.551429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909541cc-df49-4cc7-a0e4-30b4c5f5fd33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:16.554891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909541cc-df49-4cc7-a0e4-30b4c5f5fd33 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:16.575140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 691/1388] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:49:19.759582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173b052-de23-41d4-80a1-ae85112d7e48 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:19.760181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173b052-de23-41d4-80a1-ae85112d7e48 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:19.760277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173b052-de23-41d4-80a1-ae85112d7e48 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:19.761871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 698/1389] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:19.770326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29feab-7ee7-43be-a2ec-0d27259cc49e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:19.771187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29feab-7ee7-43be-a2ec-0d27259cc49e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:19.771365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29feab-7ee7-43be-a2ec-0d27259cc49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:19.794765 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bb29feab-7ee7-43be-a2ec-0d27259cc49e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:19.796669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bb29feab-7ee7-43be-a2ec-0d27259cc49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:19.808419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 692/1390] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:19.875766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173b052-de23-41d4-80a1-ae85112d7e48 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:19.905478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173b052-de23-41d4-80a1-ae85112d7e48 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:19.915815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47aa1a-4f44-4b95-8ee0-a4350ddbae7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:19.916950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47aa1a-4f44-4b95-8ee0-a4350ddbae7d service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:19.917391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47aa1a-4f44-4b95-8ee0-a4350ddbae7d service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:19.917789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47aa1a-4f44-4b95-8ee0-a4350ddbae7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:19.935201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47aa1a-4f44-4b95-8ee0-a4350ddbae7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:19.938502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47aa1a-4f44-4b95-8ee0-a4350ddbae7d service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:19.959226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 699/1391] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:49:23.205505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe63b6b-777c-4aa0-a740-59820df1022a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:23.206664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe63b6b-777c-4aa0-a740-59820df1022a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:23.207123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe63b6b-777c-4aa0-a740-59820df1022a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:23.210303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 693/1392] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:23.219501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dec40-1ab9-4a11-ab3c-ddc72b3d2cbc None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:23.220408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dec40-1ab9-4a11-ab3c-ddc72b3d2cbc None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:23.220408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dec40-1ab9-4a11-ab3c-ddc72b3d2cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:23.255128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-435dec40-1ab9-4a11-ab3c-ddc72b3d2cbc None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:23.257333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-435dec40-1ab9-4a11-ab3c-ddc72b3d2cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:23.273471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 700/1393] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:23.326781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe63b6b-777c-4aa0-a740-59820df1022a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:23.345989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe63b6b-777c-4aa0-a740-59820df1022a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:23.352548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37849ee6-57b5-43e2-a368-5059c750d92c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:23.353245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849ee6-57b5-43e2-a368-5059c750d92c service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:23.353518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849ee6-57b5-43e2-a368-5059c750d92c service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:23.353797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37849ee6-57b5-43e2-a368-5059c750d92c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:23.363940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37849ee6-57b5-43e2-a368-5059c750d92c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:23.366426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37849ee6-57b5-43e2-a368-5059c750d92c service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:23.379598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 694/1394] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:49:25.066542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c2fa8-c681-4d03-bc45-2d49b970d4b0 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:25.067111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c2fa8-c681-4d03-bc45-2d49b970d4b0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:25.067238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c2fa8-c681-4d03-bc45-2d49b970d4b0 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:25.069607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 701/1395] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:25.076471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f2cb7-3c32-49cc-9554-5c3478be99e6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:25.077315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f2cb7-3c32-49cc-9554-5c3478be99e6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:25.077315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f2cb7-3c32-49cc-9554-5c3478be99e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:25.087926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=52 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:25.101517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-cf6f2cb7-3c32-49cc-9554-5c3478be99e6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:25.102811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cf6f2cb7-3c32-49cc-9554-5c3478be99e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:25.115834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 695/1396] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:49:25.182342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2693b-b225-424c-b5ce-1397b3f39658 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:25.182841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2693b-b225-424c-b5ce-1397b3f39658 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:25.183313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2693b-b225-424c-b5ce-1397b3f39658 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:25.185679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 702/1397] 10.222.0.72 () {54 vars in 962 bytes} [Thu Jun 16 14:49:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:25.192234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4cb56-1e50-4ec6-888c-f02ffc74a43a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:25.192982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4cb56-1e50-4ec6-888c-f02ffc74a43a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:25.193432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4cb56-1e50-4ec6-888c-f02ffc74a43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:25.289896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b8e4cb56-1e50-4ec6-888c-f02ffc74a43a None None] MFA Rules not processed for user `0239f0ca3ab3423491d3844fd79a2868`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:25.291982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b8e4cb56-1e50-4ec6-888c-f02ffc74a43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:25.377446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 696/1398] 10.222.0.72 () {58 vars in 1059 bytes} [Thu Jun 16 14:49:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:25.386078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2693b-b225-424c-b5ce-1397b3f39658 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:25.393132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:25.402111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2693b-b225-424c-b5ce-1397b3f39658 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:25.409594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3e5ca-2059-4612-81dc-785a95e21919 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:25.411820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e5ca-2059-4612-81dc-785a95e21919 service glance] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:25.412153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e5ca-2059-4612-81dc-785a95e21919 service glance] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:25.412442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e5ca-2059-4612-81dc-785a95e21919 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:25.423085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3e5ca-2059-4612-81dc-785a95e21919 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:25.424563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3e5ca-2059-4612-81dc-785a95e21919 service glance] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:25.435951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 703/1399] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:49:26.335421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4e9bd-1950-4e0b-b17d-36017c70ce42 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:26.335984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4e9bd-1950-4e0b-b17d-36017c70ce42 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:26.336647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4e9bd-1950-4e0b-b17d-36017c70ce42 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:26.340211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 697/1400] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:26.350104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c3c87-e4a6-4fd3-bd9e-eb5624cfcd30 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:26.350924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c3c87-e4a6-4fd3-bd9e-eb5624cfcd30 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:26.351077 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c3c87-e4a6-4fd3-bd9e-eb5624cfcd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:26.364456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefc672-2924-4a95-ac1e-5965c8e7dcb5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:26.364754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefc672-2924-4a95-ac1e-5965c8e7dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:26.365847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefc672-2924-4a95-ac1e-5965c8e7dcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:26.387580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0d2c3c87-e4a6-4fd3-bd9e-eb5624cfcd30 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:26.389585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0d2c3c87-e4a6-4fd3-bd9e-eb5624cfcd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:26.403257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 704/1401] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:26.451851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-feefc672-2924-4a95-ac1e-5965c8e7dcb5 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:26.453549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-feefc672-2924-4a95-ac1e-5965c8e7dcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:26.463730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c3c87-e4a6-4fd3-bd9e-eb5624cfcd30 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:26.480543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c3c87-e4a6-4fd3-bd9e-eb5624cfcd30 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:26.488401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02688fb0-787f-4924-8d85-e8f0daf1fcbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:26.489413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02688fb0-787f-4924-8d85-e8f0daf1fcbb service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:26.489826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02688fb0-787f-4924-8d85-e8f0daf1fcbb service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:26.490249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02688fb0-787f-4924-8d85-e8f0daf1fcbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:26.505507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02688fb0-787f-4924-8d85-e8f0daf1fcbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:26.509218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02688fb0-787f-4924-8d85-e8f0daf1fcbb service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:26.529302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 705/1402] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:49:26.552903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 698/1403] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:49:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:49:26.625680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cd245-5ad9-4377-b4d5-01aa0f0e7996 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:26.627811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cd245-5ad9-4377-b4d5-01aa0f0e7996 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:26.627811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cd245-5ad9-4377-b4d5-01aa0f0e7996 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:26.629206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 706/1404] 10.222.0.72 () {54 vars in 942 bytes} [Thu Jun 16 14:49:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:26.637487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a681a-6617-4018-855b-570574175a0b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:26.637913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a681a-6617-4018-855b-570574175a0b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:26.638287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a681a-6617-4018-855b-570574175a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:26.737817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-aa9a681a-6617-4018-855b-570574175a0b None None] MFA Rules not processed for user `2013d4199e3841109b7c4661ff45b160`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:26.740263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-aa9a681a-6617-4018-855b-570574175a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:26.767042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:26.859911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 699/1405] 10.222.0.72 () {58 vars in 1039 bytes} [Thu Jun 16 14:49:26 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:49:26.875583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cd245-5ad9-4377-b4d5-01aa0f0e7996 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:26.898101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cd245-5ad9-4377-b4d5-01aa0f0e7996 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:26.904498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd6908-f2f2-4c2e-9b66-d76cf190a478 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:26.905388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6908-f2f2-4c2e-9b66-d76cf190a478 service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:26.905736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6908-f2f2-4c2e-9b66-d76cf190a478 service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:26.906093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd6908-f2f2-4c2e-9b66-d76cf190a478 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:26.917105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd6908-f2f2-4c2e-9b66-d76cf190a478 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:26.919337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd6908-f2f2-4c2e-9b66-d76cf190a478 service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:26.933813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 707/1406] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 14:49:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 14:49:29.295427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd0b97-8ba4-414c-b134-2a5c226a9455 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:29.296064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd0b97-8ba4-414c-b134-2a5c226a9455 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:29.296064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd0b97-8ba4-414c-b134-2a5c226a9455 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:29.297751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 700/1407] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:29.305926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49134975-1220-4040-ad3d-97a1ceb71a1a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:29.306770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49134975-1220-4040-ad3d-97a1ceb71a1a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:29.306955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49134975-1220-4040-ad3d-97a1ceb71a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:29.332486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-49134975-1220-4040-ad3d-97a1ceb71a1a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:29.333906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-49134975-1220-4040-ad3d-97a1ceb71a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:29.343873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 708/1408] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:49:29.411106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd0b97-8ba4-414c-b134-2a5c226a9455 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:49:29.469064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd0b97-8ba4-414c-b134-2a5c226a9455 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:49:29.482943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5d12-b2f1-4e39-ad8d-e68ecdadd837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:49:29.485443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5d12-b2f1-4e39-ad8d-e68ecdadd837 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:29.485443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5d12-b2f1-4e39-ad8d-e68ecdadd837 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:29.485443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5d12-b2f1-4e39-ad8d-e68ecdadd837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:29.497893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5d12-b2f1-4e39-ad8d-e68ecdadd837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:49:29.502933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5d12-b2f1-4e39-ad8d-e68ecdadd837 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:49:29.513468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 701/1409] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:49:39.328775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=23 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:39.506446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=30 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:49:56.104412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0229b14-77e9-4008-8f72-c57e2afd0203 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:56.105129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0229b14-77e9-4008-8f72-c57e2afd0203 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:56.105496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0229b14-77e9-4008-8f72-c57e2afd0203 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:56.108229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 709/1410] 10.222.0.72 () {54 vars in 918 bytes} [Thu Jun 16 14:49:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:49:56.116827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e229c-6c2d-4f90-be94-8dff1e1f06ef None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:56.117738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e229c-6c2d-4f90-be94-8dff1e1f06ef None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:56.117738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e229c-6c2d-4f90-be94-8dff1e1f06ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:56.159725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-019e229c-6c2d-4f90-be94-8dff1e1f06ef None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:56.161357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-019e229c-6c2d-4f90-be94-8dff1e1f06ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:56.171298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 702/1411] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Jun 16 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:49:56.196407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a566eb-ad44-40f2-b28e-5aeac5cbe59d None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:56.196759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a566eb-ad44-40f2-b28e-5aeac5cbe59d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:56.197157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a566eb-ad44-40f2-b28e-5aeac5cbe59d None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:56.198789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 710/1412] 10.222.0.72 () {54 vars in 918 bytes} [Thu Jun 16 14:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:49:56.209589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f031-f3f3-4166-8ee0-48fe1db93188 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:49:56.209998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f031-f3f3-4166-8ee0-48fe1db93188 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:49:56.210493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752f031-f3f3-4166-8ee0-48fe1db93188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:49:56.242908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2752f031-f3f3-4166-8ee0-48fe1db93188 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:49:56.245199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2752f031-f3f3-4166-8ee0-48fe1db93188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:49:56.261613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 703/1413] 10.222.0.72 () {58 vars in 1015 bytes} [Thu Jun 16 14:49:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:00.385444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d373ad7-3ff2-4954-b5d3-3b3977fb9cde None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:00.386571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d373ad7-3ff2-4954-b5d3-3b3977fb9cde None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:00.386571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d373ad7-3ff2-4954-b5d3-3b3977fb9cde None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:00.387424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:00.397056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfc07-799b-40ee-8e76-d1153ba8ccb9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:00.397811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfc07-799b-40ee-8e76-d1153ba8ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:00.397946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfc07-799b-40ee-8e76-d1153ba8ccb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:00.428774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9d0bfc07-799b-40ee-8e76-d1153ba8ccb9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:00.431056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9d0bfc07-799b-40ee-8e76-d1153ba8ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:00.445397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:00.492104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d373ad7-3ff2-4954-b5d3-3b3977fb9cde None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:00.508250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d373ad7-3ff2-4954-b5d3-3b3977fb9cde None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:00.516160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e8d92-9820-48be-a58e-436a83fad583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:00.516912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e8d92-9820-48be-a58e-436a83fad583 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:00.517238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e8d92-9820-48be-a58e-436a83fad583 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:00.517562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e8d92-9820-48be-a58e-436a83fad583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:00.527828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e8d92-9820-48be-a58e-436a83fad583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:00.529552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e8d92-9820-48be-a58e-436a83fad583 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:00.541358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 712/1416] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:50:01.659089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c773c8-7981-4b35-b8e6-99fea3dbbecb None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:01.659888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c773c8-7981-4b35-b8e6-99fea3dbbecb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:01.659888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c773c8-7981-4b35-b8e6-99fea3dbbecb None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:01.661296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:01.668341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d1064-8fba-4753-8d86-cdeaaf0930bc None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:01.670474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d1064-8fba-4753-8d86-cdeaaf0930bc None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:01.670474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d1064-8fba-4753-8d86-cdeaaf0930bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:01.704628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fe5d1064-8fba-4753-8d86-cdeaaf0930bc None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:01.707114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fe5d1064-8fba-4753-8d86-cdeaaf0930bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:01.722872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:01.914237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c773c8-7981-4b35-b8e6-99fea3dbbecb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:01.930123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c773c8-7981-4b35-b8e6-99fea3dbbecb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:01.936875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd75629-9fa3-48a5-8a7f-5155c83a6f3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:01.937585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd75629-9fa3-48a5-8a7f-5155c83a6f3c service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:01.937908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd75629-9fa3-48a5-8a7f-5155c83a6f3c service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:01.938192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd75629-9fa3-48a5-8a7f-5155c83a6f3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:01.950723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd75629-9fa3-48a5-8a7f-5155c83a6f3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:01.952403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd75629-9fa3-48a5-8a7f-5155c83a6f3c service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:01.963523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 706/1419] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:50:02.997252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1a226-5cb9-43c4-866a-bfb91253f873 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:02.997859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1a226-5cb9-43c4-866a-bfb91253f873 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:02.997859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1a226-5cb9-43c4-866a-bfb91253f873 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:02.999270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:03.008715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29273f-22c3-4456-a6f2-351ea55e139a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:03.009420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29273f-22c3-4456-a6f2-351ea55e139a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:03.009845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29273f-22c3-4456-a6f2-351ea55e139a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:03.041772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1d29273f-22c3-4456-a6f2-351ea55e139a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:03.044283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1d29273f-22c3-4456-a6f2-351ea55e139a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:03.055595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:03.215450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1a226-5cb9-43c4-866a-bfb91253f873 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:03.231808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1a226-5cb9-43c4-866a-bfb91253f873 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:03.239793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae6daa-0261-4696-af57-0f62110bda3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:03.240598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae6daa-0261-4696-af57-0f62110bda3f service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:03.241031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae6daa-0261-4696-af57-0f62110bda3f service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:03.241435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae6daa-0261-4696-af57-0f62110bda3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:03.253547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae6daa-0261-4696-af57-0f62110bda3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:03.257104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae6daa-0261-4696-af57-0f62110bda3f service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:03.276784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 715/1422] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:04.284636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e75bbc-92d9-4020-a9af-38a34c26e7b6 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:04.285551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e75bbc-92d9-4020-a9af-38a34c26e7b6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:04.285713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e75bbc-92d9-4020-a9af-38a34c26e7b6 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:04.287579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:50:04.293896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893717c-92a4-4968-a4ac-de1beba783a7 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:04.294543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893717c-92a4-4968-a4ac-de1beba783a7 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:04.294543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893717c-92a4-4968-a4ac-de1beba783a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:04.314521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-e893717c-92a4-4968-a4ac-de1beba783a7 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:04.315770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e893717c-92a4-4968-a4ac-de1beba783a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:04.324785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:04.489389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e75bbc-92d9-4020-a9af-38a34c26e7b6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:04.514827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e75bbc-92d9-4020-a9af-38a34c26e7b6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:04.524454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd0f4e-8871-4af5-9a3e-95d47cb70d09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:04.525586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd0f4e-8871-4af5-9a3e-95d47cb70d09 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:04.526087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd0f4e-8871-4af5-9a3e-95d47cb70d09 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:04.526583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd0f4e-8871-4af5-9a3e-95d47cb70d09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:04.542958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd0f4e-8871-4af5-9a3e-95d47cb70d09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:04.545686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd0f4e-8871-4af5-9a3e-95d47cb70d09 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:04.561788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 709/1425] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:06.120559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a09a-abac-470d-ab03-afc817e38f0f None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.121250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a09a-abac-470d-ab03-afc817e38f0f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.121250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610a09a-abac-470d-ab03-afc817e38f0f None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.122879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:06.131281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4bfc6-9a87-4bf6-a7ed-2c0873b47d32 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.132172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4bfc6-9a87-4bf6-a7ed-2c0873b47d32 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.132392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4bfc6-9a87-4bf6-a7ed-2c0873b47d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.163527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-81e4bfc6-9a87-4bf6-a7ed-2c0873b47d32 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:06.165698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-81e4bfc6-9a87-4bf6-a7ed-2c0873b47d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:06.180218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:06.349258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610a09a-abac-470d-ab03-afc817e38f0f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:06.364934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610a09a-abac-470d-ab03-afc817e38f0f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:06.371084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd18821-5a86-4946-809c-bc012857993e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:06.371668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18821-5a86-4946-809c-bc012857993e service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:06.371957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18821-5a86-4946-809c-bc012857993e service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:06.372241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18821-5a86-4946-809c-bc012857993e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:06.382246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd18821-5a86-4946-809c-bc012857993e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:06.383958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd18821-5a86-4946-809c-bc012857993e service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:06.396134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 718/1428] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:07.331143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838fa0-8adc-408c-a1f3-cbdd4e2569df None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.331979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838fa0-8adc-408c-a1f3-cbdd4e2569df None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.332515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838fa0-8adc-408c-a1f3-cbdd4e2569df None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.335758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:07.343584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32a1b-37bf-4c97-9862-c8684ca47a02 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.344418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32a1b-37bf-4c97-9862-c8684ca47a02 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.344418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e32a1b-37bf-4c97-9862-c8684ca47a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.367027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-93e32a1b-37bf-4c97-9862-c8684ca47a02 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:07.368270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-93e32a1b-37bf-4c97-9862-c8684ca47a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:07.376710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:07.539873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d838fa0-8adc-408c-a1f3-cbdd4e2569df None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:07.556636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d838fa0-8adc-408c-a1f3-cbdd4e2569df None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:07.563202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0951b-71e9-4897-9243-5379068e7a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:07.563866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0951b-71e9-4897-9243-5379068e7a8f service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:07.564154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0951b-71e9-4897-9243-5379068e7a8f service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:07.564444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0951b-71e9-4897-9243-5379068e7a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:07.574205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0951b-71e9-4897-9243-5379068e7a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:07.575848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea0951b-71e9-4897-9243-5379068e7a8f service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:07.587303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 712/1431] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:50:08.711221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7924395-1628-4941-b15c-9163537eaf71 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.711979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7924395-1628-4941-b15c-9163537eaf71 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.711979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7924395-1628-4941-b15c-9163537eaf71 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.713337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:08.722302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f9c2-7f9e-4725-baba-3a26518cf119 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.722905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f9c2-7f9e-4725-baba-3a26518cf119 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.722905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f9c2-7f9e-4725-baba-3a26518cf119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.748218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-55f7f9c2-7f9e-4725-baba-3a26518cf119 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:08.749515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-55f7f9c2-7f9e-4725-baba-3a26518cf119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:08.758804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:08.921072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7924395-1628-4941-b15c-9163537eaf71 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:08.941808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7924395-1628-4941-b15c-9163537eaf71 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:08.948364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91fc59-b1ba-4317-9130-413fe1098317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:08.949215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91fc59-b1ba-4317-9130-413fe1098317 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:08.949590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91fc59-b1ba-4317-9130-413fe1098317 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:08.949956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91fc59-b1ba-4317-9130-413fe1098317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:08.960778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91fc59-b1ba-4317-9130-413fe1098317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:08.962536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91fc59-b1ba-4317-9130-413fe1098317 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:08.974087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 721/1434] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:50:09.892255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e217a-6894-4e07-bdd1-5d1c41d8f38e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:09.892899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e217a-6894-4e07-bdd1-5d1c41d8f38e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:09.892899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e217a-6894-4e07-bdd1-5d1c41d8f38e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:09.894572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:09.900882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f0e8-4a6a-4642-82b9-18d1e1866679 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:09.901779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f0e8-4a6a-4642-82b9-18d1e1866679 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:09.901779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242f0e8-4a6a-4642-82b9-18d1e1866679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:09.923463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-6242f0e8-4a6a-4642-82b9-18d1e1866679 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:09.924712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6242f0e8-4a6a-4642-82b9-18d1e1866679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:09.933710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:50:10.106199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e217a-6894-4e07-bdd1-5d1c41d8f38e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:10.125623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e217a-6894-4e07-bdd1-5d1c41d8f38e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:10.132980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d2157-79fd-41eb-9acc-0b8d29afb06b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:10.133899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d2157-79fd-41eb-9acc-0b8d29afb06b service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:10.134253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d2157-79fd-41eb-9acc-0b8d29afb06b service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:10.134663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d2157-79fd-41eb-9acc-0b8d29afb06b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:10.150505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d2157-79fd-41eb-9acc-0b8d29afb06b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:10.152562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d2157-79fd-41eb-9acc-0b8d29afb06b service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:10.165984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 715/1437] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:11.326765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1a10-9ec2-4335-98e7-0ecd05ec37f1 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:11.327401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1a10-9ec2-4335-98e7-0ecd05ec37f1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:11.327401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1a10-9ec2-4335-98e7-0ecd05ec37f1 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:11.328915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:11.335606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e257-272e-4ffe-95ed-a05fcc08daf6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:11.336295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e257-272e-4ffe-95ed-a05fcc08daf6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:11.336295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e257-272e-4ffe-95ed-a05fcc08daf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:11.359103 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1581e257-272e-4ffe-95ed-a05fcc08daf6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:11.360357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1581e257-272e-4ffe-95ed-a05fcc08daf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:11.369710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:11.654520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1a10-9ec2-4335-98e7-0ecd05ec37f1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:11.670035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1a10-9ec2-4335-98e7-0ecd05ec37f1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:11.676003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287fcbd9-18d9-439c-ab9d-7da3d7632433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:11.677372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fcbd9-18d9-439c-ab9d-7da3d7632433 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:11.677702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fcbd9-18d9-439c-ab9d-7da3d7632433 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:11.678058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287fcbd9-18d9-439c-ab9d-7da3d7632433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:11.688019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fcbd9-18d9-439c-ab9d-7da3d7632433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:11.690008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287fcbd9-18d9-439c-ab9d-7da3d7632433 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:11.702076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 724/1440] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:50:13.905632 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff03c-7137-4570-baf9-1f28bd963b58 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.906472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff03c-7137-4570-baf9-1f28bd963b58 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.906642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ff03c-7137-4570-baf9-1f28bd963b58 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.908194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:13.915621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76cc43-2b53-4149-9b8e-fd71da614484 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:13.916259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76cc43-2b53-4149-9b8e-fd71da614484 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:13.916659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76cc43-2b53-4149-9b8e-fd71da614484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:13.948287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-4e76cc43-2b53-4149-9b8e-fd71da614484 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:13.949953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4e76cc43-2b53-4149-9b8e-fd71da614484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:13.962428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:14.009654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ff03c-7137-4570-baf9-1f28bd963b58 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:14.026037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ff03c-7137-4570-baf9-1f28bd963b58 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:14.032657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdd4a9-cfd4-4c55-ab34-64f8fb44627e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:14.033711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdd4a9-cfd4-4c55-ab34-64f8fb44627e service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:14.034154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdd4a9-cfd4-4c55-ab34-64f8fb44627e service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:14.034595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdd4a9-cfd4-4c55-ab34-64f8fb44627e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:14.051255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdd4a9-cfd4-4c55-ab34-64f8fb44627e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:14.054020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdd4a9-cfd4-4c55-ab34-64f8fb44627e service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:14.072041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 718/1443] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:17.007225 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbe31b-c97c-437d-ac5a-762dbbdf2bb4 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.008041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbe31b-c97c-437d-ac5a-762dbbdf2bb4 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.008041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbe31b-c97c-437d-ac5a-762dbbdf2bb4 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.009966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:17.016172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a6a15-03d7-4cf0-be7a-92835bcdef82 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.016736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a6a15-03d7-4cf0-be7a-92835bcdef82 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.016736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a6a15-03d7-4cf0-be7a-92835bcdef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.036931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9b5a6a15-03d7-4cf0-be7a-92835bcdef82 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:17.038176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9b5a6a15-03d7-4cf0-be7a-92835bcdef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:17.046667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:17.087644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbe31b-c97c-437d-ac5a-762dbbdf2bb4 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:17.099700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbe31b-c97c-437d-ac5a-762dbbdf2bb4 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:17.103347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd46c2-f324-4b02-9126-e4a8d9b63ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:17.104122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd46c2-f324-4b02-9126-e4a8d9b63ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:17.104452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd46c2-f324-4b02-9126-e4a8d9b63ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:17.104774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd46c2-f324-4b02-9126-e4a8d9b63ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:17.115817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd46c2-f324-4b02-9126-e4a8d9b63ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:17.117331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd46c2-f324-4b02-9126-e4a8d9b63ae4 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:17.128499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 727/1446] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:19.949937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b1b1-8675-4291-a440-d32f0aafa86d None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.950800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b1b1-8675-4291-a440-d32f0aafa86d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.950800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b1b1-8675-4291-a440-d32f0aafa86d None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.952297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:19.960560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206ecb5-2e63-4993-b267-4a7a13792bce None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:19.961426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206ecb5-2e63-4993-b267-4a7a13792bce None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:19.961619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206ecb5-2e63-4993-b267-4a7a13792bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:19.988111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8206ecb5-2e63-4993-b267-4a7a13792bce None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:19.989955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8206ecb5-2e63-4993-b267-4a7a13792bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:19.999295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:20.070178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b1b1-8675-4291-a440-d32f0aafa86d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:20.077115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=55 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:20.085065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b1b1-8675-4291-a440-d32f0aafa86d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:20.089601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339a290-e33e-4416-b9f4-bf1950b3055f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:20.090278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339a290-e33e-4416-b9f4-bf1950b3055f service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:20.090561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339a290-e33e-4416-b9f4-bf1950b3055f service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:20.090837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339a290-e33e-4416-b9f4-bf1950b3055f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:20.102355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339a290-e33e-4416-b9f4-bf1950b3055f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:20.103810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339a290-e33e-4416-b9f4-bf1950b3055f service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:20.116492 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 721/1449] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:50:22.979133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab32a15-f7ff-48a2-8ee3-e24473eb7f3d None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:22.980046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab32a15-f7ff-48a2-8ee3-e24473eb7f3d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:22.980046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab32a15-f7ff-48a2-8ee3-e24473eb7f3d None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:22.981741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:22.987840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d1f2-9caa-4121-9ac3-9c0715fff1df None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:22.988626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d1f2-9caa-4121-9ac3-9c0715fff1df None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:22.988626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d1f2-9caa-4121-9ac3-9c0715fff1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.013549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f165d1f2-9caa-4121-9ac3-9c0715fff1df None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:23.015336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f165d1f2-9caa-4121-9ac3-9c0715fff1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:23.027613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:50:23.081495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab32a15-f7ff-48a2-8ee3-e24473eb7f3d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:23.098939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab32a15-f7ff-48a2-8ee3-e24473eb7f3d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:23.102872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7d893-ebe6-4f63-8bae-a37cc2efbd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:23.103572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7d893-ebe6-4f63-8bae-a37cc2efbd3f service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:23.103862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7d893-ebe6-4f63-8bae-a37cc2efbd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:23.104142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7d893-ebe6-4f63-8bae-a37cc2efbd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:23.115928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7d893-ebe6-4f63-8bae-a37cc2efbd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:23.117470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7d893-ebe6-4f63-8bae-a37cc2efbd3f service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:23.128607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 730/1452] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:26.031649 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8234-7961-488e-b27f-20ea6007444e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.032544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8234-7961-488e-b27f-20ea6007444e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.032925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8234-7961-488e-b27f-20ea6007444e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.034703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:26.042765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2e179-92f4-4dab-869c-4a210066078f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.043493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2e179-92f4-4dab-869c-4a210066078f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.043991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2e179-92f4-4dab-869c-4a210066078f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.069848 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-52d2e179-92f4-4dab-869c-4a210066078f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:26.071105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-52d2e179-92f4-4dab-869c-4a210066078f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:26.081330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:26.126177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8234-7961-488e-b27f-20ea6007444e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:26.147544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8234-7961-488e-b27f-20ea6007444e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:26.154467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805039f-ee44-4c1c-acd9-427333921aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:26.155585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805039f-ee44-4c1c-acd9-427333921aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:26.156095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805039f-ee44-4c1c-acd9-427333921aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:26.156596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805039f-ee44-4c1c-acd9-427333921aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:26.176187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805039f-ee44-4c1c-acd9-427333921aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:26.178781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805039f-ee44-4c1c-acd9-427333921aa2 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:26.196077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 724/1455] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:29.051621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc56f4-cdb3-40fa-821f-735fc7933d66 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:29.052451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc56f4-cdb3-40fa-821f-735fc7933d66 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:29.052451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc56f4-cdb3-40fa-821f-735fc7933d66 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:29.054147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:29.060433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29454-00f8-4989-b5ef-98d96a348c15 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:29.061104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29454-00f8-4989-b5ef-98d96a348c15 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:29.061104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29454-00f8-4989-b5ef-98d96a348c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:29.084310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c6c29454-00f8-4989-b5ef-98d96a348c15 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:29.085653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c6c29454-00f8-4989-b5ef-98d96a348c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:29.094763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:50:29.135460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc56f4-cdb3-40fa-821f-735fc7933d66 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:29.148277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc56f4-cdb3-40fa-821f-735fc7933d66 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:29.151815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce020a8d-c8ec-4fcb-81b0-3de313ac8449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:29.152587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce020a8d-c8ec-4fcb-81b0-3de313ac8449 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:29.152966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce020a8d-c8ec-4fcb-81b0-3de313ac8449 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:29.153177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce020a8d-c8ec-4fcb-81b0-3de313ac8449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:29.164138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce020a8d-c8ec-4fcb-81b0-3de313ac8449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:29.165715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce020a8d-c8ec-4fcb-81b0-3de313ac8449 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:29.177590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 733/1458] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:31.623349 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b6db-a7dc-4a27-9976-167bc4844771 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.624072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b6db-a7dc-4a27-9976-167bc4844771 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.624072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2b6db-a7dc-4a27-9976-167bc4844771 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.625746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:31.632638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06c23c-6948-4f61-8bef-087676cc1b0f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.633381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06c23c-6948-4f61-8bef-087676cc1b0f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.633381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06c23c-6948-4f61-8bef-087676cc1b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.639993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=70 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:31.656030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8c06c23c-6948-4f61-8bef-087676cc1b0f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:31.657368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8c06c23c-6948-4f61-8bef-087676cc1b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:31.667114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:31.708820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2b6db-a7dc-4a27-9976-167bc4844771 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:31.721500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2b6db-a7dc-4a27-9976-167bc4844771 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:31.725437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8d5e2-d3e3-4c60-9748-39c5766a40cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:31.726141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8d5e2-d3e3-4c60-9748-39c5766a40cc service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:31.726435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8d5e2-d3e3-4c60-9748-39c5766a40cc service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:31.726707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8d5e2-d3e3-4c60-9748-39c5766a40cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:31.737681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8d5e2-d3e3-4c60-9748-39c5766a40cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:31.740118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8d5e2-d3e3-4c60-9748-39c5766a40cc service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:31.752282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 727/1461] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:34.148120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdfb62-d9d5-4f85-952b-e0e33bf937ef None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:34.148708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdfb62-d9d5-4f85-952b-e0e33bf937ef None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:34.148708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdfb62-d9d5-4f85-952b-e0e33bf937ef None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:34.150016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:34.155810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bcc0-cd4e-4a26-8077-32172ae4434f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:34.156465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bcc0-cd4e-4a26-8077-32172ae4434f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:34.156465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bcc0-cd4e-4a26-8077-32172ae4434f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:34.175490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9080bcc0-cd4e-4a26-8077-32172ae4434f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:34.177325 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9080bcc0-cd4e-4a26-8077-32172ae4434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:34.187023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:34.229581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdfb62-d9d5-4f85-952b-e0e33bf937ef None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:34.248492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdfb62-d9d5-4f85-952b-e0e33bf937ef None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:34.255490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7a931-83d9-404f-b51a-c3e7be226240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:34.256544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a931-83d9-404f-b51a-c3e7be226240 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:34.257024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a931-83d9-404f-b51a-c3e7be226240 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:34.257525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7a931-83d9-404f-b51a-c3e7be226240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:34.275788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a931-83d9-404f-b51a-c3e7be226240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:34.278702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7a931-83d9-404f-b51a-c3e7be226240 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:34.297642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 736/1464] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:36.614344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c577b8-ab48-4fae-bfeb-9618fcff8207 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.615047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c577b8-ab48-4fae-bfeb-9618fcff8207 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.615047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c577b8-ab48-4fae-bfeb-9618fcff8207 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.616594 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:36.623052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf670-9c96-4ba7-a7a3-1cc7430ae4c9 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.623674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf670-9c96-4ba7-a7a3-1cc7430ae4c9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.623674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27bf670-9c96-4ba7-a7a3-1cc7430ae4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.645993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a27bf670-9c96-4ba7-a7a3-1cc7430ae4c9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:36.647686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a27bf670-9c96-4ba7-a7a3-1cc7430ae4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:36.657701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:50:36.698229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c577b8-ab48-4fae-bfeb-9618fcff8207 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:36.711599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c577b8-ab48-4fae-bfeb-9618fcff8207 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:36.715934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828b49-7152-4b5f-837a-37b24767beb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:36.716658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b49-7152-4b5f-837a-37b24767beb3 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:36.716993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b49-7152-4b5f-837a-37b24767beb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:36.717275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b49-7152-4b5f-837a-37b24767beb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:36.728827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db828b49-7152-4b5f-837a-37b24767beb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:36.730450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db828b49-7152-4b5f-837a-37b24767beb3 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:36.742966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 730/1467] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:50:39.145331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57429d1-ce60-484f-b944-0b45b556e06f None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.145902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57429d1-ce60-484f-b944-0b45b556e06f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.145902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57429d1-ce60-484f-b944-0b45b556e06f None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.147500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:50:39.153637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea80f1c-89dc-4efa-9592-647b7489123d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.154211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea80f1c-89dc-4efa-9592-647b7489123d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.154211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea80f1c-89dc-4efa-9592-647b7489123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.173183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0ea80f1c-89dc-4efa-9592-647b7489123d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:50:39.174411 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0ea80f1c-89dc-4efa-9592-647b7489123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:50:39.182952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:50:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:50:39.222001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57429d1-ce60-484f-b944-0b45b556e06f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:50:39.234688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57429d1-ce60-484f-b944-0b45b556e06f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:50:39.238408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81321b2f-ee4b-4817-ac7c-395e67296e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:50:39.239212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81321b2f-ee4b-4817-ac7c-395e67296e32 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:50:39.239498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81321b2f-ee4b-4817-ac7c-395e67296e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:50:39.240124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81321b2f-ee4b-4817-ac7c-395e67296e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:50:39.250707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81321b2f-ee4b-4817-ac7c-395e67296e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:50:39.252217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81321b2f-ee4b-4817-ac7c-395e67296e32 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:50:39.262948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 739/1470] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:50:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:50:49.167445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=35 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:50:49.258332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=17 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:02.944221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe4824-51ea-42be-9224-63027b363931 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.945381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe4824-51ea-42be-9224-63027b363931 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.945381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe4824-51ea-42be-9224-63027b363931 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.946370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 732/1471] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:02.955197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9a7e-90a6-4c82-ad61-a696da1a03fd None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:02.955872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9a7e-90a6-4c82-ad61-a696da1a03fd None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:02.956092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9a7e-90a6-4c82-ad61-a696da1a03fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:02.983585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c40d9a7e-90a6-4c82-ad61-a696da1a03fd None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:02.985737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c40d9a7e-90a6-4c82-ad61-a696da1a03fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:02.999893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 740/1472] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:03.084959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2519bc-361c-497a-991d-956e96797265 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.085327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2519bc-361c-497a-991d-956e96797265 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.085685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2519bc-361c-497a-991d-956e96797265 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.087480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 733/1473] 10.222.0.72 () {54 vars in 962 bytes} [Thu Jun 16 14:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:03.094923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc09818-3b6c-4035-a3ae-95a27a91e618 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.095436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc09818-3b6c-4035-a3ae-95a27a91e618 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.095939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc09818-3b6c-4035-a3ae-95a27a91e618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.127633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5fc09818-3b6c-4035-a3ae-95a27a91e618 None None] MFA Rules not processed for user `0239f0ca3ab3423491d3844fd79a2868`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:03.129928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5fc09818-3b6c-4035-a3ae-95a27a91e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:03.142949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 741/1474] 10.222.0.72 () {58 vars in 1059 bytes} [Thu Jun 16 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:03.157480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2519bc-361c-497a-991d-956e96797265 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:03.176101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2519bc-361c-497a-991d-956e96797265 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:03.183551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941d005-5e6b-45d6-a806-1f1e6a0ba04b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:03.184610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941d005-5e6b-45d6-a806-1f1e6a0ba04b service glance] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:03.185156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941d005-5e6b-45d6-a806-1f1e6a0ba04b service glance] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:03.185656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941d005-5e6b-45d6-a806-1f1e6a0ba04b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:03.202753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941d005-5e6b-45d6-a806-1f1e6a0ba04b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:03.205660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941d005-5e6b-45d6-a806-1f1e6a0ba04b service glance] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:03.225410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 734/1475] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:51:05.994749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028dd74-6f5d-42e4-a5b3-7f3868dde46a None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:05.995618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028dd74-6f5d-42e4-a5b3-7f3868dde46a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:05.995618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028dd74-6f5d-42e4-a5b3-7f3868dde46a None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:05.997279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 742/1476] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:06.005316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5477a2-c24c-41fb-8cfe-2e06f0133543 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.005999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5477a2-c24c-41fb-8cfe-2e06f0133543 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.006326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5477a2-c24c-41fb-8cfe-2e06f0133543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.040295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ee5477a2-c24c-41fb-8cfe-2e06f0133543 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:06.042992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ee5477a2-c24c-41fb-8cfe-2e06f0133543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:06.064208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 735/1477] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:06.091239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028dd74-6f5d-42e4-a5b3-7f3868dde46a None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:06.105152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028dd74-6f5d-42e4-a5b3-7f3868dde46a None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:06.110382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9c068-e134-4672-a72d-9aade598a034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:06.111139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c068-e134-4672-a72d-9aade598a034 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:06.111424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c068-e134-4672-a72d-9aade598a034 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:06.111702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9c068-e134-4672-a72d-9aade598a034 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:06.113085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9c068-e134-4672-a72d-9aade598a034 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:06.114897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-c2e9c068-e134-4672-a72d-9aade598a034 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:51:06.114897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:51:06.114897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:51:06.116097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9c068-e134-4672-a72d-9aade598a034 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:06.139802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 743/1478] 10.222.0.72 () {56 vars in 1108 bytes} [Thu Jun 16 14:51:06 2022] GET /identity/v3/projects => generated 2553 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:51:08.512447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba444b9-b9cc-4f92-b511-6facb9a7e6b9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:08.513327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba444b9-b9cc-4f92-b511-6facb9a7e6b9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:08.513327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba444b9-b9cc-4f92-b511-6facb9a7e6b9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:08.514655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 736/1479] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:08.524023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb89d0-4f4f-408c-a748-53bd0ac2508b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:08.524899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb89d0-4f4f-408c-a748-53bd0ac2508b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:08.524899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb89d0-4f4f-408c-a748-53bd0ac2508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:08.545221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-40bb89d0-4f4f-408c-a748-53bd0ac2508b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:08.546486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-40bb89d0-4f4f-408c-a748-53bd0ac2508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:08.555885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 744/1480] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:08.608062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba444b9-b9cc-4f92-b511-6facb9a7e6b9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:08.631830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba444b9-b9cc-4f92-b511-6facb9a7e6b9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:08.640944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a236cc-9d2a-48bb-add4-376d86886126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:08.641666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a236cc-9d2a-48bb-add4-376d86886126 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:08.642041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a236cc-9d2a-48bb-add4-376d86886126 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:08.642326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a236cc-9d2a-48bb-add4-376d86886126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:08.654436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a236cc-9d2a-48bb-add4-376d86886126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:08.656307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a236cc-9d2a-48bb-add4-376d86886126 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:08.669583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 737/1481] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:51:11.234668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4a2df-d950-4615-b051-8252b4f45251 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.235272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4a2df-d950-4615-b051-8252b4f45251 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.235272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4a2df-d950-4615-b051-8252b4f45251 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.236683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 745/1482] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:11.245158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d89030-2551-45b6-8de5-3f6a413e7abc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.245854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d89030-2551-45b6-8de5-3f6a413e7abc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.245854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d89030-2551-45b6-8de5-3f6a413e7abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.266043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a3d89030-2551-45b6-8de5-3f6a413e7abc None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:11.267437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a3d89030-2551-45b6-8de5-3f6a413e7abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:11.277398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 738/1483] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:11.337412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4a2df-d950-4615-b051-8252b4f45251 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:11.354109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4a2df-d950-4615-b051-8252b4f45251 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:11.358520 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81e29c-ed51-4ad3-9ddc-8a34bc7a6515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:11.359244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81e29c-ed51-4ad3-9ddc-8a34bc7a6515 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:11.359565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81e29c-ed51-4ad3-9ddc-8a34bc7a6515 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:11.359886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81e29c-ed51-4ad3-9ddc-8a34bc7a6515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:11.371939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81e29c-ed51-4ad3-9ddc-8a34bc7a6515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:11.373809 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81e29c-ed51-4ad3-9ddc-8a34bc7a6515 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:11.387045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 746/1484] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:51:14.179145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1b81-e7e0-4520-a9ed-0d435a1bc127 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:14.180058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1b81-e7e0-4520-a9ed-0d435a1bc127 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:14.180937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd1b81-e7e0-4520-a9ed-0d435a1bc127 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:14.184880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 739/1485] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:51:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:14.194806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87810ff2-c15d-4c1b-ad64-05b38bf1fde6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:14.195448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87810ff2-c15d-4c1b-ad64-05b38bf1fde6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:14.195823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87810ff2-c15d-4c1b-ad64-05b38bf1fde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:14.225146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-87810ff2-c15d-4c1b-ad64-05b38bf1fde6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:14.226992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-87810ff2-c15d-4c1b-ad64-05b38bf1fde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:14.239925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 747/1486] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:14.302912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd1b81-e7e0-4520-a9ed-0d435a1bc127 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:14.323732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd1b81-e7e0-4520-a9ed-0d435a1bc127 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:14.330410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf70f-74cd-441e-8560-af889e8b4245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:14.332094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf70f-74cd-441e-8560-af889e8b4245 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:14.332601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf70f-74cd-441e-8560-af889e8b4245 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:14.333156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf70f-74cd-441e-8560-af889e8b4245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:14.351544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbf70f-74cd-441e-8560-af889e8b4245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:14.354382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbf70f-74cd-441e-8560-af889e8b4245 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:14.375436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 740/1487] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:51:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:51:18.058664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc1fd6-2c3e-4c5c-8449-9e774f7c3396 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.059439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc1fd6-2c3e-4c5c-8449-9e774f7c3396 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.059617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc1fd6-2c3e-4c5c-8449-9e774f7c3396 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.062299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 748/1488] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:51:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:51:18.071909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c1d2-4cd6-4ecb-9ed0-4b763acc8d75 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.072727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c1d2-4cd6-4ecb-9ed0-4b763acc8d75 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.072956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c1d2-4cd6-4ecb-9ed0-4b763acc8d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.098808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a766c1d2-4cd6-4ecb-9ed0-4b763acc8d75 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:18.100239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a766c1d2-4cd6-4ecb-9ed0-4b763acc8d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:18.115302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 741/1489] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:18.163512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc1fd6-2c3e-4c5c-8449-9e774f7c3396 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:18.181570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc1fd6-2c3e-4c5c-8449-9e774f7c3396 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:18.187578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e20b8-aca7-4ef8-b9b8-cdf1e53a20df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:18.188401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e20b8-aca7-4ef8-b9b8-cdf1e53a20df service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:18.188874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e20b8-aca7-4ef8-b9b8-cdf1e53a20df service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:18.189328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e20b8-aca7-4ef8-b9b8-cdf1e53a20df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:18.206877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e20b8-aca7-4ef8-b9b8-cdf1e53a20df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:18.209420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e20b8-aca7-4ef8-b9b8-cdf1e53a20df service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:18.226836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 749/1490] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:51:20.910969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c31089-4888-48bf-b570-f606c9b9dbb9 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.911522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c31089-4888-48bf-b570-f606c9b9dbb9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.911522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c31089-4888-48bf-b570-f606c9b9dbb9 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.912956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 742/1491] 10.222.0.72 () {54 vars in 926 bytes} [Thu Jun 16 14:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:51:20.919401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cffd-0574-43f3-ac01-9cb376f557f8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:20.920164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cffd-0574-43f3-ac01-9cb376f557f8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:20.920164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0cffd-0574-43f3-ac01-9cb376f557f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:20.940712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-37f0cffd-0574-43f3-ac01-9cb376f557f8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:51:20.942078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-37f0cffd-0574-43f3-ac01-9cb376f557f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:51:20.951435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 750/1492] 10.222.0.72 () {58 vars in 1023 bytes} [Thu Jun 16 14:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:51:20.994057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c31089-4888-48bf-b570-f606c9b9dbb9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:51:21.006048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c31089-4888-48bf-b570-f606c9b9dbb9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:51:21.010320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d51b8-3ca4-49d9-98fe-a138ec9fd979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:51:21.011009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d51b8-3ca4-49d9-98fe-a138ec9fd979 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:51:21.011285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d51b8-3ca4-49d9-98fe-a138ec9fd979 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:51:21.011600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d51b8-3ca4-49d9-98fe-a138ec9fd979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:51:21.024203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d51b8-3ca4-49d9-98fe-a138ec9fd979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:51:21.025976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d51b8-3ca4-49d9-98fe-a138ec9fd979 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:51:21.038636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 743/1493] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:51:30.934588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=33 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:51:31.033282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=33 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:10.026041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71992b-5382-4390-9264-70ac4e4c4bc9 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.026782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71992b-5382-4390-9264-70ac4e4c4bc9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.026782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71992b-5382-4390-9264-70ac4e4c4bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.047550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ce71992b-5382-4390-9264-70ac4e4c4bc9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:10.048796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ce71992b-5382-4390-9264-70ac4e4c4bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:10.057748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 751/1494] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:53:10.068686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a908c-3ebd-4509-9e7f-7eeb0118de24 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.069342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a908c-3ebd-4509-9e7f-7eeb0118de24 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.069620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a908c-3ebd-4509-9e7f-7eeb0118de24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.092441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-228a908c-3ebd-4509-9e7f-7eeb0118de24 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:10.094449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-228a908c-3ebd-4509-9e7f-7eeb0118de24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:10.111405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 744/1495] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:53:10.113092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71992b-5382-4390-9264-70ac4e4c4bc9 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.128319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71992b-5382-4390-9264-70ac4e4c4bc9 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.134085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.134778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.135059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.135347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.136094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:53:10.136401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.137567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:10.137567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:53:10.137567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:53:10.138411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.154653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 752/1496] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:53:10.165068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a908c-3ebd-4509-9e7f-7eeb0118de24 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.175508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a908c-3ebd-4509-9e7f-7eeb0118de24 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.181031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.181810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.182098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.182367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.183064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.184969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:10.184969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: warnings.warn(msg) Jun 16 14:53:10.184969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: Jun 16 14:53:10.186195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.233104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 745/1497] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:53:10 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:53:10.239669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.252928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb41b73-25a2-4acd-b534-bbfebdef6b13 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.325200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.325891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.326333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.326618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.327357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.329269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:10.329269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:53:10.329269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:53:10.330521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.358861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 753/1498] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:53:10 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:53:10.364973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.379136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d041d4-018a-48b7-8f82-fded53a13da0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.384697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.385532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.385894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.386219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.387054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.388317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:10.388317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: warnings.warn(msg) Jun 16 14:53:10.388317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: Jun 16 14:53:10.389374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.406136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 746/1499] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:53:10.410652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.422869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae68ffa-380b-409e-abae-387894611de3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.428700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd34f7-c700-4198-9b73-cf3f48ed8b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.429438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd34f7-c700-4198-9b73-cf3f48ed8b29 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.429825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd34f7-c700-4198-9b73-cf3f48ed8b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.430109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd34f7-c700-4198-9b73-cf3f48ed8b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.430868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd34f7-c700-4198-9b73-cf3f48ed8b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.431986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-11fd34f7-c700-4198-9b73-cf3f48ed8b29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:10.431986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:53:10.431986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:53:10.433226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd34f7-c700-4198-9b73-cf3f48ed8b29 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.453888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 754/1500] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:53:10.459175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.475895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6747f2-7af2-434d-b14f-c6b88c62a412 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.483140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d27a10-bc43-4e02-bf37-cbeeb4ee8220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.485201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d27a10-bc43-4e02-bf37-cbeeb4ee8220 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.485635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d27a10-bc43-4e02-bf37-cbeeb4ee8220 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.486090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d27a10-bc43-4e02-bf37-cbeeb4ee8220 admin admin] PATH_INFO: `/v3/projects/c97258423f1e4c3f8f382972acf4e5e3/users/0f89f19c8aeb4a4e986972c9ecf26dc9/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.505793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d27a10-bc43-4e02-bf37-cbeeb4ee8220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c97258423f1e4c3f8f382972acf4e5e3, user_id=0f89f19c8aeb4a4e986972c9ecf26dc9, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.507019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-73d27a10-bc43-4e02-bf37-cbeeb4ee8220 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:10.507019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: warnings.warn(msg) Jun 16 14:53:10.507019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: Jun 16 14:53:10.509636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d27a10-bc43-4e02-bf37-cbeeb4ee8220 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.526950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 747/1501] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:53:10 2022] PUT /identity/v3/projects/c97258423f1e4c3f8f382972acf4e5e3/users/0f89f19c8aeb4a4e986972c9ecf26dc9/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:53:10.539412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70c725-d436-4fb5-b3c9-f1c5a59ca575 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.539779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70c725-d436-4fb5-b3c9-f1c5a59ca575 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.539983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70c725-d436-4fb5-b3c9-f1c5a59ca575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.574526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-1a70c725-d436-4fb5-b3c9-f1c5a59ca575 None None] MFA Rules not processed for user `0f89f19c8aeb4a4e986972c9ecf26dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:10.575904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1a70c725-d436-4fb5-b3c9-f1c5a59ca575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:10.657084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 755/1502] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:53:10.678683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33b12c-baa2-42f3-8da9-cfe232e193d0 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.678958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33b12c-baa2-42f3-8da9-cfe232e193d0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.679234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33b12c-baa2-42f3-8da9-cfe232e193d0 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.680650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 748/1503] 10.222.0.72 () {54 vars in 963 bytes} [Thu Jun 16 14:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:10.686357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf374c-e7ef-4610-9f56-a51e057f43d4 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.687394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf374c-e7ef-4610-9f56-a51e057f43d4 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.687902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf374c-e7ef-4610-9f56-a51e057f43d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.707658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-faaf374c-e7ef-4610-9f56-a51e057f43d4 None None] MFA Rules not processed for user `6812dddc3db746b6893f12f1ae0c715d`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:10.710232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-faaf374c-e7ef-4610-9f56-a51e057f43d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:10.768698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:10.780516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 756/1504] 10.222.0.72 () {58 vars in 1060 bytes} [Thu Jun 16 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:53:10.789347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33b12c-baa2-42f3-8da9-cfe232e193d0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.801375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33b12c-baa2-42f3-8da9-cfe232e193d0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.805233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.805951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.806273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.806587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.817414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.819175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:10.830404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 749/1505] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:53:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:53:10.881237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8994467-3d14-4eca-a85b-3faec8401018 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.881679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8994467-3d14-4eca-a85b-3faec8401018 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.882112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8994467-3d14-4eca-a85b-3faec8401018 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.883922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 757/1506] 10.222.0.72 () {52 vars in 829 bytes} [Thu Jun 16 14:53:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:53:10.888807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:10.966489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d5a6d-2086-43ec-8e88-a6d252b9ad73 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:10.974655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:10.976170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:10.976665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:10.977229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] PATH_INFO: `/v3/users/0f89f19c8aeb4a4e986972c9ecf26dc9` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:10.979517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f89f19c8aeb4a4e986972c9ecf26dc9)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:10.981473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:10.981473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: warnings.warn(msg) Jun 16 14:53:10.981473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: Jun 16 14:53:10.983335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:11.091142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f89f19c8aeb4a4e986972c9ecf26dc9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:11.092198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f89f19c8aeb4a4e986972c9ecf26dc9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:11.110578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f89f19c8aeb4a4e986972c9ecf26dc9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:11.111497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f89f19c8aeb4a4e986972c9ecf26dc9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:11.122934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f89f19c8aeb4a4e986972c9ecf26dc9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:11.144342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:11.159123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 750/1507] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:53:10 2022] DELETE /identity/v3/users/0f89f19c8aeb4a4e986972c9ecf26dc9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:53:11.165917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:11.166564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:11.166934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:11.198021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:53:11.200095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:11.308049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 758/1508] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:53:11.320248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:11.321906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:11.396656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877e95d-faa2-4f0e-89be-6304b2504182 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:11.401323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec08a71-bac9-4df3-afba-01b8c8169d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:11.402074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec08a71-bac9-4df3-afba-01b8c8169d88 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:11.402403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec08a71-bac9-4df3-afba-01b8c8169d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:11.402727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec08a71-bac9-4df3-afba-01b8c8169d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:11.414552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec08a71-bac9-4df3-afba-01b8c8169d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:11.416288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec08a71-bac9-4df3-afba-01b8c8169d88 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:11.428437 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 751/1509] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:53:11.868650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:53:11.870997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:53:11.884619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5333a-505b-4415-8e3f-2ad6edd9358f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:53:11.893126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:53:11.894040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:53:11.894316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:53:11.894589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] PATH_INFO: `/v3/projects/c97258423f1e4c3f8f382972acf4e5e3` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:53:11.895945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c97258423f1e4c3f8f382972acf4e5e3)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:53:11.897115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:53:11.897115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:53:11.897115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:53:11.898429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:53:11.978234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c97258423f1e4c3f8f382972acf4e5e3', 'request_id': 'req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:11.985728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c97258423f1e4c3f8f382972acf4e5e3', 'request_id': 'req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:12.000721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c97258423f1e4c3f8f382972acf4e5e3', 'request_id': 'req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:12.022972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] The token cache is being invalidate because project c97258423f1e4c3f8f382972acf4e5e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:53:12.023259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:12.024316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6bb8af38-8eb2-4afc-8df6-3c3dfdc8cfcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:53:12.025949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 759/1510] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:53:11 2022] DELETE /identity/v3/projects/c97258423f1e4c3f8f382972acf4e5e3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:53:21.423360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:53:22.019432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:01.563375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588106a5-96e6-4137-9735-f76e958e1ead None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:01.563983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588106a5-96e6-4137-9735-f76e958e1ead None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:01.563983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588106a5-96e6-4137-9735-f76e958e1ead None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:01.565415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:01.570732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2ed9c-c1b4-4f0f-a64b-295b66c5d836 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:01.571284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2ed9c-c1b4-4f0f-a64b-295b66c5d836 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:01.571284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2ed9c-c1b4-4f0f-a64b-295b66c5d836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:01.588820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0ad2ed9c-c1b4-4f0f-a64b-295b66c5d836 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:01.590041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0ad2ed9c-c1b4-4f0f-a64b-295b66c5d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:01.657654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:55:01.832785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588106a5-96e6-4137-9735-f76e958e1ead None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:01.834841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-588106a5-96e6-4137-9735-f76e958e1ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:01.910740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588106a5-96e6-4137-9735-f76e958e1ead None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:01.916119 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6233fbc-9914-4fbb-b33a-82d27c1fa090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:01.916817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233fbc-9914-4fbb-b33a-82d27c1fa090 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:01.917155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233fbc-9914-4fbb-b33a-82d27c1fa090 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:01.917457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233fbc-9914-4fbb-b33a-82d27c1fa090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:01.926412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6233fbc-9914-4fbb-b33a-82d27c1fa090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:01.928016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6233fbc-9914-4fbb-b33a-82d27c1fa090 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:01.937441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 753/1513] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:02.819792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e990a3-a6cd-45b7-86dd-5f83a6eb0c77 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:02.820382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e990a3-a6cd-45b7-86dd-5f83a6eb0c77 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:02.820382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e990a3-a6cd-45b7-86dd-5f83a6eb0c77 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:02.821779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:02.826722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61cc55-3f3b-4f3a-8c60-7c17a5c2be71 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:02.827191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61cc55-3f3b-4f3a-8c60-7c17a5c2be71 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:02.827305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61cc55-3f3b-4f3a-8c60-7c17a5c2be71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:02.845525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4f61cc55-3f3b-4f3a-8c60-7c17a5c2be71 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:02.846729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4f61cc55-3f3b-4f3a-8c60-7c17a5c2be71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:02.854830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:55:03.009141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e990a3-a6cd-45b7-86dd-5f83a6eb0c77 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:03.023528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e990a3-a6cd-45b7-86dd-5f83a6eb0c77 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:03.029376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2018f-476f-4044-9650-a7967bb9a9a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:03.030024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2018f-476f-4044-9650-a7967bb9a9a9 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:03.030310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2018f-476f-4044-9650-a7967bb9a9a9 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:03.030589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2018f-476f-4044-9650-a7967bb9a9a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:03.039220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2018f-476f-4044-9650-a7967bb9a9a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:03.040825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2018f-476f-4044-9650-a7967bb9a9a9 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:03.051448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 762/1516] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:04.596623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebdc16-4409-4802-a45b-1f9d3114ddab None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:04.597252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebdc16-4409-4802-a45b-1f9d3114ddab None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:04.597252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebdc16-4409-4802-a45b-1f9d3114ddab None None] PATH_INFO: `/v3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:04.598519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 755/1517] 10.222.0.86 () {54 vars in 925 bytes} [Thu Jun 16 14:55:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:55:04.605485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511580b-29b1-4ecd-961a-c733b3420705 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:04.605931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511580b-29b1-4ecd-961a-c733b3420705 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:04.606046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511580b-29b1-4ecd-961a-c733b3420705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:04.624985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7511580b-29b1-4ecd-961a-c733b3420705 None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:04.626208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7511580b-29b1-4ecd-961a-c733b3420705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:04.691254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 763/1518] 10.222.0.86 () {58 vars in 1015 bytes} [Thu Jun 16 14:55:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:55:12.840230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=37 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:13.904646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068b3e-7437-4454-b07a-607d44c95cd4 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:13.905333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068b3e-7437-4454-b07a-607d44c95cd4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:13.905333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068b3e-7437-4454-b07a-607d44c95cd4 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:13.907423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 756/1519] 10.222.0.72 () {54 vars in 960 bytes} [Thu Jun 16 14:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:13.913269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce39b79-5b6a-4b60-834c-291433cb823c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:13.913806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce39b79-5b6a-4b60-834c-291433cb823c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:13.914102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce39b79-5b6a-4b60-834c-291433cb823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:13.936935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5ce39b79-5b6a-4b60-834c-291433cb823c None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:13.938203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5ce39b79-5b6a-4b60-834c-291433cb823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:13.947333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 764/1520] 10.222.0.72 () {58 vars in 1057 bytes} [Thu Jun 16 14:55:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:55:13.955666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a068b3e-7437-4454-b07a-607d44c95cd4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:13.970940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a068b3e-7437-4454-b07a-607d44c95cd4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:13.976406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355b6aa-43bf-4ec7-8341-2834a1a9461c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:13.977088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355b6aa-43bf-4ec7-8341-2834a1a9461c service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:13.977376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355b6aa-43bf-4ec7-8341-2834a1a9461c service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:13.977654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355b6aa-43bf-4ec7-8341-2834a1a9461c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:13.987330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355b6aa-43bf-4ec7-8341-2834a1a9461c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:13.989010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355b6aa-43bf-4ec7-8341-2834a1a9461c service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:14.000604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 757/1521] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:14.453909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51a01-7a38-4b60-aff9-6e3547a1b66c None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:14.454187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51a01-7a38-4b60-aff9-6e3547a1b66c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:14.454466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51a01-7a38-4b60-aff9-6e3547a1b66c None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:14.456112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:55:14.461627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1f9a-922e-4c30-bb17-3c6d05d55883 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:14.461962 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1f9a-922e-4c30-bb17-3c6d05d55883 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:14.462345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1f9a-922e-4c30-bb17-3c6d05d55883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:14.481268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ab4d1f9a-922e-4c30-bb17-3c6d05d55883 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:14.482507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ab4d1f9a-922e-4c30-bb17-3c6d05d55883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:14.490535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:55:14.651422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b51a01-7a38-4b60-aff9-6e3547a1b66c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:14.667418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b51a01-7a38-4b60-aff9-6e3547a1b66c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:14.676083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe8ffd-b3e4-4db1-86bc-0aaa3b731daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:14.676740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe8ffd-b3e4-4db1-86bc-0aaa3b731daf service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:14.677084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe8ffd-b3e4-4db1-86bc-0aaa3b731daf service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:14.677362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe8ffd-b3e4-4db1-86bc-0aaa3b731daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:14.690493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe8ffd-b3e4-4db1-86bc-0aaa3b731daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:14.692870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe8ffd-b3e4-4db1-86bc-0aaa3b731daf service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:14.713808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 766/1524] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:15.621507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9338c-b5a4-4c53-9520-8c29eae041be None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:15.622083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9338c-b5a4-4c53-9520-8c29eae041be None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:15.622083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9338c-b5a4-4c53-9520-8c29eae041be None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:15.624172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 14:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:55:15.629430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee219b6c-ddd6-4319-bb44-0b527d9200e1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:15.629947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee219b6c-ddd6-4319-bb44-0b527d9200e1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:15.630056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee219b6c-ddd6-4319-bb44-0b527d9200e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:15.648120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ee219b6c-ddd6-4319-bb44-0b527d9200e1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:15.649375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ee219b6c-ddd6-4319-bb44-0b527d9200e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:15.657532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 14:55:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:55:15.814212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9338c-b5a4-4c53-9520-8c29eae041be None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:15.829341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9338c-b5a4-4c53-9520-8c29eae041be None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:15.834790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa41f8-3e43-484e-8afc-851796886ff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:15.835434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa41f8-3e43-484e-8afc-851796886ff1 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:15.835710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa41f8-3e43-484e-8afc-851796886ff1 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:15.835983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa41f8-3e43-484e-8afc-851796886ff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:15.845099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa41f8-3e43-484e-8afc-851796886ff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:15.846693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa41f8-3e43-484e-8afc-851796886ff1 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:15.856651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 760/1527] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:20.095318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115374-b2c7-4a88-b84c-c8d862c3cebe None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:20.095907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115374-b2c7-4a88-b84c-c8d862c3cebe None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:20.095907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98115374-b2c7-4a88-b84c-c8d862c3cebe None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:20.097289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 14:55:20.105347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49338a59-51b1-4dc1-89cb-55a72b5c9286 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:20.105896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49338a59-51b1-4dc1-89cb-55a72b5c9286 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:20.106244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49338a59-51b1-4dc1-89cb-55a72b5c9286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:20.127468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-49338a59-51b1-4dc1-89cb-55a72b5c9286 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:20.128680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-49338a59-51b1-4dc1-89cb-55a72b5c9286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:20.136881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:55:20.175426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98115374-b2c7-4a88-b84c-c8d862c3cebe None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:20.177119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-98115374-b2c7-4a88-b84c-c8d862c3cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:20.238159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98115374-b2c7-4a88-b84c-c8d862c3cebe None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:20.242066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ecead7-2995-4a41-8e3e-52a6bb8ff0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:20.242713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecead7-2995-4a41-8e3e-52a6bb8ff0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:20.242996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecead7-2995-4a41-8e3e-52a6bb8ff0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:20.243270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ecead7-2995-4a41-8e3e-52a6bb8ff0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:20.252831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecead7-2995-4a41-8e3e-52a6bb8ff0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:20.254301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ecead7-2995-4a41-8e3e-52a6bb8ff0a5 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:20.259383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:20.266205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 769/1530] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:22.595525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba384fce-a0de-4b6d-b7d8-d9d6bf2ad01f None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:22.596079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba384fce-a0de-4b6d-b7d8-d9d6bf2ad01f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:22.596079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba384fce-a0de-4b6d-b7d8-d9d6bf2ad01f None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:22.597477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:22.603491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e23d7-3e47-4963-b916-34cb70217ef6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:22.604063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e23d7-3e47-4963-b916-34cb70217ef6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:22.604063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e23d7-3e47-4963-b916-34cb70217ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:22.624404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0b6e23d7-3e47-4963-b916-34cb70217ef6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:22.625673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0b6e23d7-3e47-4963-b916-34cb70217ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:22.634487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:55:22.672135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba384fce-a0de-4b6d-b7d8-d9d6bf2ad01f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:22.683183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba384fce-a0de-4b6d-b7d8-d9d6bf2ad01f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:22.687283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f00c-1c38-4ad6-ae27-74c8587168c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:22.687973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f00c-1c38-4ad6-ae27-74c8587168c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:22.688290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f00c-1c38-4ad6-ae27-74c8587168c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:22.688537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f00c-1c38-4ad6-ae27-74c8587168c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:22.698150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6f00c-1c38-4ad6-ae27-74c8587168c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:22.699827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6f00c-1c38-4ad6-ae27-74c8587168c0 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:22.711451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 763/1533] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:55:27.036194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c70c-225f-4bbc-a188-2a7d451f8643 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:27.036810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c70c-225f-4bbc-a188-2a7d451f8643 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:27.036810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c70c-225f-4bbc-a188-2a7d451f8643 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:27.038317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:27.044156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540dfe-8ba9-40e1-b98e-9753ace3a074 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:27.044623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540dfe-8ba9-40e1-b98e-9753ace3a074 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:27.044724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22540dfe-8ba9-40e1-b98e-9753ace3a074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:27.063592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-22540dfe-8ba9-40e1-b98e-9753ace3a074 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:27.064805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-22540dfe-8ba9-40e1-b98e-9753ace3a074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:27.073080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:55:27.380533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c70c-225f-4bbc-a188-2a7d451f8643 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:27.382250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2159c70c-225f-4bbc-a188-2a7d451f8643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:27.444775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c70c-225f-4bbc-a188-2a7d451f8643 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:27.448317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f6a23-0b27-4aa2-831a-0bcdcdb39a63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:27.449001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f6a23-0b27-4aa2-831a-0bcdcdb39a63 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:27.449295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f6a23-0b27-4aa2-831a-0bcdcdb39a63 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:27.449567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f6a23-0b27-4aa2-831a-0bcdcdb39a63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:27.459666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f6a23-0b27-4aa2-831a-0bcdcdb39a63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:27.461224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f6a23-0b27-4aa2-831a-0bcdcdb39a63 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:27.471721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 772/1536] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:29.680815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf5fd2-82e3-4453-b245-42bbd5d9c44a None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:29.681428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf5fd2-82e3-4453-b245-42bbd5d9c44a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:29.681428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf5fd2-82e3-4453-b245-42bbd5d9c44a None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:29.682786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 14:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:55:29.688963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feeda9-8183-40d6-a04a-b220339a4321 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:29.689514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feeda9-8183-40d6-a04a-b220339a4321 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:29.689514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98feeda9-8183-40d6-a04a-b220339a4321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:29.708746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-98feeda9-8183-40d6-a04a-b220339a4321 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:55:29.710022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-98feeda9-8183-40d6-a04a-b220339a4321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:29.718173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 14:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:55:29.988716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf5fd2-82e3-4453-b245-42bbd5d9c44a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:55:29.990464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dbaf5fd2-82e3-4453-b245-42bbd5d9c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:55:30.003392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf5fd2-82e3-4453-b245-42bbd5d9c44a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:55:30.007091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30dae0b-8f34-47d6-ad30-537c4bd64ad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:55:30.007722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dae0b-8f34-47d6-ad30-537c4bd64ad0 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:55:30.008003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dae0b-8f34-47d6-ad30-537c4bd64ad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:55:30.008278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30dae0b-8f34-47d6-ad30-537c4bd64ad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:55:30.018189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dae0b-8f34-47d6-ad30-537c4bd64ad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:55:30.019619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30dae0b-8f34-47d6-ad30-537c4bd64ad0 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:55:30.030894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 766/1539] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:55:39.703387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=32 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:55:40.025537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=33 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:01.877867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491e76b-d078-4509-b5b5-697da03f4ad0 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:01.878567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491e76b-d078-4509-b5b5-697da03f4ad0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:01.879773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491e76b-d078-4509-b5b5-697da03f4ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:01.931625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b491e76b-d078-4509-b5b5-697da03f4ad0 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:01.934377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b491e76b-d078-4509-b5b5-697da03f4ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:01.957491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 774/1540] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:01.968497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701424e7-f143-49a5-8293-7f5142cc38e2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:01.970155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701424e7-f143-49a5-8293-7f5142cc38e2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:01.970880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701424e7-f143-49a5-8293-7f5142cc38e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.019021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-701424e7-f143-49a5-8293-7f5142cc38e2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.021790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-701424e7-f143-49a5-8293-7f5142cc38e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.045116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 767/1541] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:02.050949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491e76b-d078-4509-b5b5-697da03f4ad0 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.073243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491e76b-d078-4509-b5b5-697da03f4ad0 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.089896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.091475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.092237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.093178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.095640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:02.096486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.100078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.127131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b2416-757f-42fb-9206-0a169c070eef None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.128950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b2416-757f-42fb-9206-0a169c070eef None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.129722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b2416-757f-42fb-9206-0a169c070eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.142552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 775/1542] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:02.153944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.193106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9050600-d41c-4ba7-ad66-dc6e298f629f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.200016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-937b2416-757f-42fb-9206-0a169c070eef None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.202485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-937b2416-757f-42fb-9206-0a169c070eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.212058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.213405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.213900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.214480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.215817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.217988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:57:02.217988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:57:02.217988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:57:02.220506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.235776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 768/1543] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:02.271850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb1555-d689-49f4-9161-4496e4f136db None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.272662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb1555-d689-49f4-9161-4496e4f136db None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.273531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb1555-d689-49f4-9161-4496e4f136db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.294435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 776/1544] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:02.298709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.335526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d3cb1555-d689-49f4-9161-4496e4f136db None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.338221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d3cb1555-d689-49f4-9161-4496e4f136db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.464572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb10a9-6e64-44ae-a3d1-1bef568b1f3c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.479435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcdba1-4fa8-453f-9f81-901f45f23cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.485776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcdba1-4fa8-453f-9f81-901f45f23cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.486726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcdba1-4fa8-453f-9f81-901f45f23cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.489383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcdba1-4fa8-453f-9f81-901f45f23cf1 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.491099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcdba1-4fa8-453f-9f81-901f45f23cf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.497479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcdba1-4fa8-453f-9f81-901f45f23cf1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.515622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 769/1545] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.524339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132d1b-a740-486f-999d-3a3192d5134d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.525364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132d1b-a740-486f-999d-3a3192d5134d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.526329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c132d1b-a740-486f-999d-3a3192d5134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.578010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 777/1546] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/users => generated 295 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:02.583335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcebd2-4f14-4e03-b2a5-766348de9c47 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.584230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcebd2-4f14-4e03-b2a5-766348de9c47 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.587273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2c132d1b-a740-486f-999d-3a3192d5134d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.587663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcebd2-4f14-4e03-b2a5-766348de9c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.594674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2c132d1b-a740-486f-999d-3a3192d5134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.599912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 770/1547] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.600945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132d1b-a740-486f-999d-3a3192d5134d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.611833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c132d1b-a740-486f-999d-3a3192d5134d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.618041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.619105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.619579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.619913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.621368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:02.621742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.622907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:57:02.622907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: warnings.warn(msg) Jun 16 14:57:02.622907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: Jun 16 14:57:02.623492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-11bcebd2-4f14-4e03-b2a5-766348de9c47 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.623947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db84b20-9cbd-4ce4-a5e7-cbc3527c3a2a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.624827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-11bcebd2-4f14-4e03-b2a5-766348de9c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.637528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 778/1548] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.638493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcebd2-4f14-4e03-b2a5-766348de9c47 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.641396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 771/1549] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:02.642883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe032a-f7b0-437a-80ce-fb42f5586516 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.643224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe032a-f7b0-437a-80ce-fb42f5586516 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.643552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe032a-f7b0-437a-80ce-fb42f5586516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.665634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcebd2-4f14-4e03-b2a5-766348de9c47 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.666482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4bfe032a-f7b0-437a-80ce-fb42f5586516 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.668493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4bfe032a-f7b0-437a-80ce-fb42f5586516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.676685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb167557-b0fe-46ec-b6e7-d4fe4cfc9c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.678060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 772/1550] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.678703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167557-b0fe-46ec-b6e7-d4fe4cfc9c2a admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.679443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11128d-81c3-47aa-b778-eaeb4e2741c3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.679743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11128d-81c3-47aa-b778-eaeb4e2741c3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.680018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11128d-81c3-47aa-b778-eaeb4e2741c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.680942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167557-b0fe-46ec-b6e7-d4fe4cfc9c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.682072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167557-b0fe-46ec-b6e7-d4fe4cfc9c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.683750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb167557-b0fe-46ec-b6e7-d4fe4cfc9c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.687176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb167557-b0fe-46ec-b6e7-d4fe4cfc9c2a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.710175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 779/1551] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:02.711505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9c409-e37a-4eb3-9ce5-14c6dab76eff None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.711796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9c409-e37a-4eb3-9ce5-14c6dab76eff None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.712073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9c409-e37a-4eb3-9ce5-14c6dab76eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.724716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1b11128d-81c3-47aa-b778-eaeb4e2741c3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.726758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1b11128d-81c3-47aa-b778-eaeb4e2741c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.736782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-94b9c409-e37a-4eb3-9ce5-14c6dab76eff None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.738317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-94b9c409-e37a-4eb3-9ce5-14c6dab76eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.741226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 773/1552] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.747567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 780/1553] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.748009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30164e-1be7-4edd-809e-d97598780f29 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.748379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9c409-e37a-4eb3-9ce5-14c6dab76eff None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.748670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30164e-1be7-4edd-809e-d97598780f29 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.749221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30164e-1be7-4edd-809e-d97598780f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.761375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9c409-e37a-4eb3-9ce5-14c6dab76eff None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.769606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d11285-166a-4a24-aa24-8563fe5ac722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.770578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d11285-166a-4a24-aa24-8563fe5ac722 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.771011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d11285-166a-4a24-aa24-8563fe5ac722 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.771426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d11285-166a-4a24-aa24-8563fe5ac722 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.772403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d11285-166a-4a24-aa24-8563fe5ac722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.774748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d11285-166a-4a24-aa24-8563fe5ac722 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.782037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8c30164e-1be7-4edd-809e-d97598780f29 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.783278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8c30164e-1be7-4edd-809e-d97598780f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.791431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 774/1554] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.792715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0211f-d056-443a-b749-243f0fa678e4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.793034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0211f-d056-443a-b749-243f0fa678e4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.793313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0211f-d056-443a-b749-243f0fa678e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.813151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 781/1555] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:02.814499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc8965-d03c-4196-b497-2a8f8a140bf8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.814791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc8965-d03c-4196-b497-2a8f8a140bf8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.815071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc8965-d03c-4196-b497-2a8f8a140bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.817379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-46d0211f-d056-443a-b749-243f0fa678e4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.818608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-46d0211f-d056-443a-b749-243f0fa678e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.834413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a6bc8965-d03c-4196-b497-2a8f8a140bf8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.835640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a6bc8965-d03c-4196-b497-2a8f8a140bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.888786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 775/1556] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.889686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0211f-d056-443a-b749-243f0fa678e4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.899410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0211f-d056-443a-b749-243f0fa678e4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.905338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673c53e-af96-40d1-8208-bc0a4f0247e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.906029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673c53e-af96-40d1-8208-bc0a4f0247e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.906350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673c53e-af96-40d1-8208-bc0a4f0247e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.906757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673c53e-af96-40d1-8208-bc0a4f0247e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.907329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673c53e-af96-40d1-8208-bc0a4f0247e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.908667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 782/1557] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.909283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673c53e-af96-40d1-8208-bc0a4f0247e7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.910381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc8965-d03c-4196-b497-2a8f8a140bf8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.918760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:02.926443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 776/1558] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:02.927966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932194ee-b3b2-4f9c-8a72-685e8b4f83b2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.928475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932194ee-b3b2-4f9c-8a72-685e8b4f83b2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.928622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932194ee-b3b2-4f9c-8a72-685e8b4f83b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.932655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc8965-d03c-4196-b497-2a8f8a140bf8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.938869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78256ea5-42c2-4fbe-a868-88fb20a40c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.939537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78256ea5-42c2-4fbe-a868-88fb20a40c1d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.939882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78256ea5-42c2-4fbe-a868-88fb20a40c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.940099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78256ea5-42c2-4fbe-a868-88fb20a40c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.940896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78256ea5-42c2-4fbe-a868-88fb20a40c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:02.941252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78256ea5-42c2-4fbe-a868-88fb20a40c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.942863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78256ea5-42c2-4fbe-a868-88fb20a40c1d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.947891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-932194ee-b3b2-4f9c-8a72-685e8b4f83b2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.949809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-932194ee-b3b2-4f9c-8a72-685e8b4f83b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.956660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 783/1559] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:02.958162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70492aa-5eca-45a1-bc4b-1ffcb721d253 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.958466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70492aa-5eca-45a1-bc4b-1ffcb721d253 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.958749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70492aa-5eca-45a1-bc4b-1ffcb721d253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.963398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 777/1560] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.965375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932194ee-b3b2-4f9c-8a72-685e8b4f83b2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:02.978559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932194ee-b3b2-4f9c-8a72-685e8b4f83b2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:02.979258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c70492aa-5eca-45a1-bc4b-1ffcb721d253 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:02.980501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c70492aa-5eca-45a1-bc4b-1ffcb721d253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:02.983261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:02.983926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:02.984384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:02.984704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:02.985485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:02.985797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:02.987467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:02.989061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 784/1561] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:02.990023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70492aa-5eca-45a1-bc4b-1ffcb721d253 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.000461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70492aa-5eca-45a1-bc4b-1ffcb721d253 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.002697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 778/1562] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:03.003614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.005251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3130bd-c7bf-4866-851b-593ed42f70af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.006030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3130bd-c7bf-4866-851b-593ed42f70af admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.006348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3130bd-c7bf-4866-851b-593ed42f70af admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.006696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3130bd-c7bf-4866-851b-593ed42f70af admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.007553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3130bd-c7bf-4866-851b-593ed42f70af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.009705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3130bd-c7bf-4866-851b-593ed42f70af admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.015090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fddb8-3f72-4648-8b26-ad9670a780b0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.022373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.023610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.024151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.024686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.026292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:03.027544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.029661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.041536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 785/1563] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:02 2022] POST /identity/v3/users => generated 284 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.042918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ee905-6556-49ae-9e2a-5239ad168b9e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.043209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ee905-6556-49ae-9e2a-5239ad168b9e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.043488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ee905-6556-49ae-9e2a-5239ad168b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.054526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 779/1564] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:03.055478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.062870 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-598ee905-6556-49ae-9e2a-5239ad168b9e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:03.063254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:03.067763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de7e30-5df8-4c97-8f5b-c9c32dae2878 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.070253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-598ee905-6556-49ae-9e2a-5239ad168b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:03.072871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.074517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.074801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.075077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] PATH_INFO: `/v3/projects/2a4c67a42f8a405d84d4697a45e962ff/users/ba40bcf173404e6c8b17d311b65bf780/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.079201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 786/1565] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:03.079991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ee905-6556-49ae-9e2a-5239ad168b9e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.089982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ee905-6556-49ae-9e2a-5239ad168b9e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.091361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4c67a42f8a405d84d4697a45e962ff, user_id=ba40bcf173404e6c8b17d311b65bf780, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.094309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.094759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d861022-98ea-4fe3-b1ad-0e6e0094af4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.099160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861022-98ea-4fe3-b1ad-0e6e0094af4c admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.099160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861022-98ea-4fe3-b1ad-0e6e0094af4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.099160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861022-98ea-4fe3-b1ad-0e6e0094af4c admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.099160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d861022-98ea-4fe3-b1ad-0e6e0094af4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.101145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d861022-98ea-4fe3-b1ad-0e6e0094af4c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.115404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 780/1566] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:03 2022] PUT /identity/v3/projects/2a4c67a42f8a405d84d4697a45e962ff/users/ba40bcf173404e6c8b17d311b65bf780/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:03.116192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.136033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 787/1567] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.136033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3aa4a-2d8f-4466-8494-75c1ff15fae8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.136033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3aa4a-2d8f-4466-8494-75c1ff15fae8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.136033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3aa4a-2d8f-4466-8494-75c1ff15fae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.163287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-bda3aa4a-2d8f-4466-8494-75c1ff15fae8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:03.164556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bda3aa4a-2d8f-4466-8494-75c1ff15fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:03.207101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf0817-afb5-4540-9216-5c2068a6e64f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.233762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 788/1568] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:03.237569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3aa4a-2d8f-4466-8494-75c1ff15fae8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.246065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3aa4a-2d8f-4466-8494-75c1ff15fae8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.250940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.251627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.251919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.252200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.252937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.254906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.276939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.277964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.278426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.278885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.280086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:03.280775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.286012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 789/1569] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.286199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.286318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.300904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 781/1570] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:03.301561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.312814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521595dd-0246-4110-98a2-5ac71758c7e3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.368511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc924df5-a92f-4b8c-a2e9-edfa0df6350f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.375014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.375712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.376002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.376282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.377062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.378726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.382029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.383153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.383588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.384004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.384725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.387321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.390579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 790/1571] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:03.391596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.402242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8addc5f-c4c3-4d9c-a582-84421a3ed195 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.407200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb04adb-b2c9-4c7a-934c-9a79ad77501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.407876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04adb-b2c9-4c7a-934c-9a79ad77501d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.408162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04adb-b2c9-4c7a-934c-9a79ad77501d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.408443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb04adb-b2c9-4c7a-934c-9a79ad77501d admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.409232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb04adb-b2c9-4c7a-934c-9a79ad77501d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:03.409559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb04adb-b2c9-4c7a-934c-9a79ad77501d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.411182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb04adb-b2c9-4c7a-934c-9a79ad77501d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.418999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:03.440872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 791/1572] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:03.440872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a89ef-068d-4d09-88f6-df69528a0fa1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.440872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a89ef-068d-4d09-88f6-df69528a0fa1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.440872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a89ef-068d-4d09-88f6-df69528a0fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.441619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 782/1573] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/projects => generated 393 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.441619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.452098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:03.452892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f38a89ef-068d-4d09-88f6-df69528a0fa1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:03.454450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f38a89ef-068d-4d09-88f6-df69528a0fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:03.519120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6f4b-9527-42ed-a971-c0288eb9384d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.522625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 792/1574] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:03.523273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a89ef-068d-4d09-88f6-df69528a0fa1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.524171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.524964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.525307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.525738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.526885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.527838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:57:03.527838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: warnings.warn(msg) Jun 16 14:57:03.527838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: Jun 16 14:57:03.529184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.537876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a89ef-068d-4d09-88f6-df69528a0fa1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.543229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.543908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.544347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.544630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.545431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:03.546056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.547611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.568577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 793/1575] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:03.568787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 783/1576] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.568961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.569071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.578372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726973e-447f-43de-a973-9decf23d37ff admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.579250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fef18d-e63c-4b00-9b49-b7d7252820b7 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.584145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.584826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.585214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.585493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.585813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.586193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.586806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.587229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.587590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.588168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.588349 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.590304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.612522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 794/1577] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/users => generated 287 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.613704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.625468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92dad2-8664-4b19-8f03-cd1d6cfebdf7 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.626983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 784/1578] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.627811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.692327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.693044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.693330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.693609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.694350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.695994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.705399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb4f45-aa8c-488e-ba2a-e582ce4cfe8f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.712021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 795/1579] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:03.712021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.712350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.712350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.712350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.712350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.712707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.714566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.728825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb792356-789f-4617-b962-ab0b7b30c9f5 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.738112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.738676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 785/1580] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.739665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.742226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.742566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.742887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.743634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.746399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.776518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 796/1581] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.780988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.804561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28df25-3c06-4625-867a-8ece47195a7a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.833231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:03.844234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:03.866120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae02062-bcb7-4f68-952a-b08c1dd728b9 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.871315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.872008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.872294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.872575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.873408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.873736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.874393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.874677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.874949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.875075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.886363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.887819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.887996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 797/1582] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:03.888746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.898971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 786/1583] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:03.899959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.900795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee8afb-9068-47b4-b7f1-2af926fd0776 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.906510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.907194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.907557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.907772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.908561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.910479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.911254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83418784-4b1c-41cd-b273-8dbda2c30aa2 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:03.917881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:03.918978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:03.919464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:03.920451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:03.921386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:03.923127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:03.938663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 787/1584] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:03.938663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.944275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 798/1585] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:03.945243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:03.950508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16a0fc9-0647-460c-9195-72398a18ab4c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.019476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5872ff-ef43-4593-8146-3a368f2cc310 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.025881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.027669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.027859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.028138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] PATH_INFO: `/v3/projects/42bcd02068ee48e7b1d5a551e943e0a1/users/8b2bb951dc0b4065bb7d95e0c350b705/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.038335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.039034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.039317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.039593 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.040380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.043011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.046644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bcd02068ee48e7b1d5a551e943e0a1, user_id=8b2bb951dc0b4065bb7d95e0c350b705, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.048332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:57:04.048332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:57:04.048332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:57:04.051153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.066884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 788/1586] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:03 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:04.068021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.068749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 799/1587] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:03 2022] PUT /identity/v3/projects/42bcd02068ee48e7b1d5a551e943e0a1/users/8b2bb951dc0b4065bb7d95e0c350b705/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:04.069537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.095194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002e1e-10dd-46e9-a5c4-54dd353bbb6b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.166329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d8154-8a98-4260-93bc-4b6f71ea3ceb admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.174242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.174952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.175242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.175517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.176233 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.178938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.183911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.184871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.185188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.185500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.186274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.187972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.204977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 789/1588] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.206146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.212969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 800/1589] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:04 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:04.212969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.221679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bc765-bf5d-469c-850c-43313f5056a2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.224646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cbf92-5ca0-430b-bf18-2505f057ee85 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.228588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.229415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.229792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.230448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.231366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.233081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.236463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.237400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.237537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.237757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.238744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.240158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.251269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 801/1590] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.252282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.258517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 790/1591] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.258517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.265324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2209f4-eaf9-4e15-bf43-65103bf2d569 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.271017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761736a-4001-45f5-8ae9-9a83b0a0fba2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.274927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5c99a-7e77-44e3-b965-e7c056f34253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.275612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5c99a-7e77-44e3-b965-e7c056f34253 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.275905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5c99a-7e77-44e3-b965-e7c056f34253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.277160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5c99a-7e77-44e3-b965-e7c056f34253 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.277796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5c99a-7e77-44e3-b965-e7c056f34253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.279721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5c99a-7e77-44e3-b965-e7c056f34253 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.282062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.284264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.285000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.286400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.290246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.295605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.299604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:04.310357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 802/1592] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:04 2022] POST /identity/v3/users => generated 303 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:04.312029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21ea6-1c9c-45a6-b483-a968c9dbfaa2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.312407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21ea6-1c9c-45a6-b483-a968c9dbfaa2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.312762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21ea6-1c9c-45a6-b483-a968c9dbfaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.325803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 791/1593] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.327501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.344155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ed0c6-2a6d-4cb5-b9d1-169b4a60bd0e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.347327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-97a21ea6-1c9c-45a6-b483-a968c9dbfaa2 None None] MFA Rules not processed for user `8b2bb951dc0b4065bb7d95e0c350b705`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:04.348682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-97a21ea6-1c9c-45a6-b483-a968c9dbfaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:04.352742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.353519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.353802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.354138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.355460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.359008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.375070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 792/1594] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.376955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.384751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:04.394372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6ecc1-9e34-4442-9bc7-37ecc9cd2778 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.401318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.403158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.403158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.403497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.403953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.405655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.424672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 793/1595] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.424672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.434394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998daef-6e3d-47ec-82e6-1025c08a53ae admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.440472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 803/1596] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:04.441471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21ea6-1c9c-45a6-b483-a968c9dbfaa2 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.444969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.446210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.446714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.447462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] PATH_INFO: `/v3/projects/420ebcc3dace46768bf717e019a47c2f/users/9fda7395f88846e4a3d8b270fb1ee564/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.456306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21ea6-1c9c-45a6-b483-a968c9dbfaa2 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.462196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.463122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.463410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.463693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] PATH_INFO: `/v3/projects/2940b81a86c2401db1aa432d9a0735eb/users/0b8be0ab53ae48af844bc281dee5f77c/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.471969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=420ebcc3dace46768bf717e019a47c2f, user_id=9fda7395f88846e4a3d8b270fb1ee564, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.474843 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.483920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2940b81a86c2401db1aa432d9a0735eb, user_id=0b8be0ab53ae48af844bc281dee5f77c, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.486833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.499934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 794/1597] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:04 2022] PUT /identity/v3/projects/420ebcc3dace46768bf717e019a47c2f/users/9fda7395f88846e4a3d8b270fb1ee564/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:04.500814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.508606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 804/1598] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:04 2022] PUT /identity/v3/projects/2940b81a86c2401db1aa432d9a0735eb/users/0b8be0ab53ae48af844bc281dee5f77c/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:04.514194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.604635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085151c-0421-4b0e-9ea1-195dac0218ac admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.613200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aaf06-28e8-4ee9-968e-2969e29dba1c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.613715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.614423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.614796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.614972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.615690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.617352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.622234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.622931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.623259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.623495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.624426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.626205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.632090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 795/1599] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.633083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.638526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 805/1600] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.640104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.646246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886d058-f501-4dcc-ad5c-3e8761dc343b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.652361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98397-aad8-440c-8441-1a918eb42d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.653590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98397-aad8-440c-8441-1a918eb42d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.653953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98397-aad8-440c-8441-1a918eb42d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.654406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98397-aad8-440c-8441-1a918eb42d7a admin admin] PATH_INFO: `/v3/projects/b017c2881aba4df6afe70cdcb84d5ee8/users/73a51a4e876e4910bbc5dd69f8a18058/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.656601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4b60-c7bd-446d-b09e-94fbb9e85098 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.674934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.674934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.674934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.674934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.675701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.675701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.677809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98397-aad8-440c-8441-1a918eb42d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b017c2881aba4df6afe70cdcb84d5ee8, user_id=73a51a4e876e4910bbc5dd69f8a18058, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.681166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98397-aad8-440c-8441-1a918eb42d7a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.693734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 806/1601] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:04.693734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.704381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 796/1602] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:04 2022] PUT /identity/v3/projects/b017c2881aba4df6afe70cdcb84d5ee8/users/73a51a4e876e4910bbc5dd69f8a18058/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:04.706799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c21aa9-3471-465b-8033-d52b6806bdcb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.707422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c21aa9-3471-465b-8033-d52b6806bdcb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.707958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c21aa9-3471-465b-8033-d52b6806bdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.741499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a3c21aa9-3471-465b-8033-d52b6806bdcb None None] MFA Rules not processed for user `9fda7395f88846e4a3d8b270fb1ee564`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:04.745810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a3c21aa9-3471-465b-8033-d52b6806bdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:04.773727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bfc84-e824-4e15-a284-25c2513af447 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:04.779240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:04.850278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:04.851251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.851549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.851835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.860930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 797/1603] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:04.865604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4b90a-7627-49ab-b97c-dcb8a4d5c70a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:04.865604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4b90a-7627-49ab-b97c-dcb8a4d5c70a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:04.865604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4b90a-7627-49ab-b97c-dcb8a4d5c70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:04.923734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-16c4b90a-7627-49ab-b97c-dcb8a4d5c70a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:04.926289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-16c4b90a-7627-49ab-b97c-dcb8a4d5c70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:04.935167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:04.937916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:04.950770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 807/1604] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:04.951455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:04.989243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:05.067725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 798/1605] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:05.069954 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4b90a-7627-49ab-b97c-dcb8a4d5c70a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:05.090112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f0816-0cc2-4e00-825b-38633b7fcd45 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:05.092878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4b90a-7627-49ab-b97c-dcb8a4d5c70a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:05.102059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:05.102754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.103921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.103921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.104339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14528862-2d9b-469c-ab4b-8dbde1952077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:05.105326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14528862-2d9b-469c-ab4b-8dbde1952077 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.105606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14528862-2d9b-469c-ab4b-8dbde1952077 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.105883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14528862-2d9b-469c-ab4b-8dbde1952077 admin admin] PATH_INFO: `/v3/projects/a0788d395b4e4632b29e263608e28973/users/1c246b5f70c54d678d827ea43d5c7a38/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.107453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:05.109106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:05.128230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 808/1606] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:05.139072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:05.145457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14528862-2d9b-469c-ab4b-8dbde1952077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0788d395b4e4632b29e263608e28973, user_id=1c246b5f70c54d678d827ea43d5c7a38, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:05.154659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14528862-2d9b-469c-ab4b-8dbde1952077 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:05.162001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450efc96-533d-44de-bd65-c5c367d1f0e7 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:05.171708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80b359-35ac-4325-b186-527f9564c543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:05.174601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80b359-35ac-4325-b186-527f9564c543 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.174894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80b359-35ac-4325-b186-527f9564c543 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.175906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80b359-35ac-4325-b186-527f9564c543 admin admin] PATH_INFO: `/v3/projects/1107ea1ea92647f8a961cb49812b53bd/users/9ce13f0b56d84e88a611733039534715/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.185743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 799/1607] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:05 2022] PUT /identity/v3/projects/a0788d395b4e4632b29e263608e28973/users/1c246b5f70c54d678d827ea43d5c7a38/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:05.188623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fac7f-51f1-41c8-86a8-1fe6a42b8d58 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.188959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fac7f-51f1-41c8-86a8-1fe6a42b8d58 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.189249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fac7f-51f1-41c8-86a8-1fe6a42b8d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.210231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80b359-35ac-4325-b186-527f9564c543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1107ea1ea92647f8a961cb49812b53bd, user_id=9ce13f0b56d84e88a611733039534715, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:05.213300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80b359-35ac-4325-b186-527f9564c543 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:05.232676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b68fac7f-51f1-41c8-86a8-1fe6a42b8d58 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:05.235378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b68fac7f-51f1-41c8-86a8-1fe6a42b8d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:05.239959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 809/1608] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:05 2022] PUT /identity/v3/projects/1107ea1ea92647f8a961cb49812b53bd/users/9ce13f0b56d84e88a611733039534715/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:05.242770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23a983-b28a-4bc1-9635-b520543d040e None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.243061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23a983-b28a-4bc1-9635-b520543d040e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.243345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23a983-b28a-4bc1-9635-b520543d040e None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.245771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 810/1609] 10.222.0.72 () {54 vars in 961 bytes} [Thu Jun 16 14:57:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:05.246682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23a983-b28a-4bc1-9635-b520543d040e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:05.248996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3b23a983-b28a-4bc1-9635-b520543d040e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:05.331334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:05.361585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 800/1610] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:05.363373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fac7f-51f1-41c8-86a8-1fe6a42b8d58 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:05.384687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fac7f-51f1-41c8-86a8-1fe6a42b8d58 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:05.388541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23a983-b28a-4bc1-9635-b520543d040e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:05.393746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdd05c-fa96-4543-a45c-cda696e2782e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:05.394852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdd05c-fa96-4543-a45c-cda696e2782e service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.395333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdd05c-fa96-4543-a45c-cda696e2782e service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.395722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdd05c-fa96-4543-a45c-cda696e2782e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.404541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119c9d0-5d1c-4b57-a982-e23a08b68dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:05.405831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119c9d0-5d1c-4b57-a982-e23a08b68dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.408126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdd05c-fa96-4543-a45c-cda696e2782e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:05.408996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119c9d0-5d1c-4b57-a982-e23a08b68dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.409777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdd05c-fa96-4543-a45c-cda696e2782e service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:05.410158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119c9d0-5d1c-4b57-a982-e23a08b68dd9 admin admin] PATH_INFO: `/v3/projects/5341374958f54cf29d9b203c0632cf7f/users/8036935d3cc547799faa68421de7ebcd/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.426594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 811/1611] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:05.428371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c3e58-c672-4d39-8dc8-2213a645a49f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.429519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c3e58-c672-4d39-8dc8-2213a645a49f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.429911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c3e58-c672-4d39-8dc8-2213a645a49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.440757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119c9d0-5d1c-4b57-a982-e23a08b68dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5341374958f54cf29d9b203c0632cf7f, user_id=8036935d3cc547799faa68421de7ebcd, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:05.451676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-df6c3e58-c672-4d39-8dc8-2213a645a49f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:05.453433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-df6c3e58-c672-4d39-8dc8-2213a645a49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:05.458705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119c9d0-5d1c-4b57-a982-e23a08b68dd9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:05.467049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 812/1612] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:05.467049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded4e1c-ec5d-4b8f-870b-2e24a7ce0d1e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.467049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded4e1c-ec5d-4b8f-870b-2e24a7ce0d1e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.467049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded4e1c-ec5d-4b8f-870b-2e24a7ce0d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.504013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 801/1613] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:05 2022] PUT /identity/v3/projects/5341374958f54cf29d9b203c0632cf7f/users/8036935d3cc547799faa68421de7ebcd/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:05.505801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5ded4e1c-ec5d-4b8f-870b-2e24a7ce0d1e None None] MFA Rules not processed for user `9ce13f0b56d84e88a611733039534715`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:05.505964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057de73-00af-40e0-a5ab-db49428f3a28 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.506103 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057de73-00af-40e0-a5ab-db49428f3a28 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.506391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057de73-00af-40e0-a5ab-db49428f3a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.507468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5ded4e1c-ec5d-4b8f-870b-2e24a7ce0d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:05.543465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c057de73-00af-40e0-a5ab-db49428f3a28 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:05.544901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c057de73-00af-40e0-a5ab-db49428f3a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:05.683955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 813/1614] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:05.690919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded4e1c-ec5d-4b8f-870b-2e24a7ce0d1e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:05.700543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 802/1615] 10.222.0.72 () {58 vars in 1058 bytes} [Thu Jun 16 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:05.704440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fad6ee-d4f3-4fb9-9ef6-ba74c55ce4f2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.709492 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fad6ee-d4f3-4fb9-9ef6-ba74c55ce4f2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.711768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fad6ee-d4f3-4fb9-9ef6-ba74c55ce4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:05.831622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c2fad6ee-d4f3-4fb9-9ef6-ba74c55ce4f2 None None] MFA Rules not processed for user `8036935d3cc547799faa68421de7ebcd`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:05.833457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c2fad6ee-d4f3-4fb9-9ef6-ba74c55ce4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:05.852184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:05.861093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded4e1c-ec5d-4b8f-870b-2e24a7ce0d1e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:05.884310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:05.884963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:05.885253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:05.885744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.045991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 803/1616] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:06.048045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fad6ee-d4f3-4fb9-9ef6-ba74c55ce4f2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.081769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fad6ee-d4f3-4fb9-9ef6-ba74c55ce4f2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.101920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5792a5-0ccf-443f-b019-2896810e8c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:06.103049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5792a5-0ccf-443f-b019-2896810e8c7e service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.103966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5792a5-0ccf-443f-b019-2896810e8c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.104108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5792a5-0ccf-443f-b019-2896810e8c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.110657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.121403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.144527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 814/1617] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:06.145742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.153789 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5792a5-0ccf-443f-b019-2896810e8c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.156509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5792a5-0ccf-443f-b019-2896810e8c7e service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.173323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07284c38-11da-4cf3-ab64-dcbb2e95dec0 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.187351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:06.190467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.190892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.191312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.219465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:06.223561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 804/1618] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:06.227184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d995cd-575f-4443-b74f-63032fb665cb None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.227485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d995cd-575f-4443-b74f-63032fb665cb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.227777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d995cd-575f-4443-b74f-63032fb665cb None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.238524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 805/1619] 10.222.0.72 () {54 vars in 961 bytes} [Thu Jun 16 14:57:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:06.239464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d995cd-575f-4443-b74f-63032fb665cb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.271263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d995cd-575f-4443-b74f-63032fb665cb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.287185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df96b4-e1fd-4b53-9de7-856d3724eb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:06.289277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df96b4-e1fd-4b53-9de7-856d3724eb3a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.291014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df96b4-e1fd-4b53-9de7-856d3724eb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.291698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df96b4-e1fd-4b53-9de7-856d3724eb3a admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.293744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df96b4-e1fd-4b53-9de7-856d3724eb3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.300997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df96b4-e1fd-4b53-9de7-856d3724eb3a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.355844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.356067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 806/1620] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:06 2022] POST /identity/v3/projects => generated 405 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:06.357887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.361686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011d826-0307-43be-8855-9fe0582db9b6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.365017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011d826-0307-43be-8855-9fe0582db9b6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.365206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011d826-0307-43be-8855-9fe0582db9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.387338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 815/1621] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:06.391830 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.394767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5011d826-0307-43be-8855-9fe0582db9b6 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:06.396562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5011d826-0307-43be-8855-9fe0582db9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:06.423896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb774b0-7d21-4e59-b9af-e4d63a0f49cd service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.545864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 807/1622] 10.222.0.72 () {58 vars in 1058 bytes} [Thu Jun 16 14:57:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:06.557643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011d826-0307-43be-8855-9fe0582db9b6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.567002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:06.567807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.568174 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.568530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.569352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.572221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.577881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011d826-0307-43be-8855-9fe0582db9b6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.584808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:06.585873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.586243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.586587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.621403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 816/1623] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:06 2022] POST /identity/v3/users => generated 309 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:06.623383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.645040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1917d-acc3-4a28-84bb-d160e7147854 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.661348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:06.662239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.663060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.663434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.664278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.666128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.699134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 817/1624] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:06.708869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.712737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.718777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.721619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9ce6c-5360-422f-a6fa-77c721f3b322 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.735016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60374f-a0bd-4d85-86b3-e55e76723653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:06.736044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60374f-a0bd-4d85-86b3-e55e76723653 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.736430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60374f-a0bd-4d85-86b3-e55e76723653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.736810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60374f-a0bd-4d85-86b3-e55e76723653 admin admin] PATH_INFO: `/v3/projects/91e0eaa896984bb8a84c069fadcb162e/users/8ee6194b42a540dfa40cd489cc7d404d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.750568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:06.764307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 808/1625] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:06.785344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60374f-a0bd-4d85-86b3-e55e76723653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e0eaa896984bb8a84c069fadcb162e, user_id=8ee6194b42a540dfa40cd489cc7d404d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:06.788013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60374f-a0bd-4d85-86b3-e55e76723653 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:06.830267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 818/1626] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:06 2022] PUT /identity/v3/projects/91e0eaa896984bb8a84c069fadcb162e/users/8ee6194b42a540dfa40cd489cc7d404d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:06.838379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:06.862191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314be43-68f3-409d-84f5-2453bbc39cd5 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:06.978955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b99f0-70b0-4c26-acc8-0615b2aeef50 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:06.979369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b99f0-70b0-4c26-acc8-0615b2aeef50 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:06.980371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b99f0-70b0-4c26-acc8-0615b2aeef50 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:06.987303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 819/1627] 10.222.0.72 () {54 vars in 912 bytes} [Thu Jun 16 14:57:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:07.014744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cd856-518c-4f3d-9eb0-c1bd8f14ade1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.017229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cd856-518c-4f3d-9eb0-c1bd8f14ade1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.017619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cd856-518c-4f3d-9eb0-c1bd8f14ade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.054570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-aa9cd856-518c-4f3d-9eb0-c1bd8f14ade1 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:07.058934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-aa9cd856-518c-4f3d-9eb0-c1bd8f14ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:07.061977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:07.064235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.064582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.065210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.067859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:07.074667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:07.096048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 809/1628] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:07.104811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:07.134739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4223d29-93ed-4239-aae2-bc6da6b77d2e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:07.146567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77bb23-8b3d-42cb-96e3-c4fc3cc7b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:07.150495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77bb23-8b3d-42cb-96e3-c4fc3cc7b819 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.151003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77bb23-8b3d-42cb-96e3-c4fc3cc7b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.151863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77bb23-8b3d-42cb-96e3-c4fc3cc7b819 admin admin] PATH_INFO: `/v3/projects/91e0eaa896984bb8a84c069fadcb162e/users/8ee6194b42a540dfa40cd489cc7d404d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.180089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77bb23-8b3d-42cb-96e3-c4fc3cc7b819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e0eaa896984bb8a84c069fadcb162e, user_id=8ee6194b42a540dfa40cd489cc7d404d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:07.238428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77bb23-8b3d-42cb-96e3-c4fc3cc7b819 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:07.241246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:07.269823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 810/1629] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:07 2022] PUT /identity/v3/projects/91e0eaa896984bb8a84c069fadcb162e/users/8ee6194b42a540dfa40cd489cc7d404d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:07.269823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c0878-c041-4364-968f-67f35c8efc54 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.269823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c0878-c041-4364-968f-67f35c8efc54 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.269823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c0878-c041-4364-968f-67f35c8efc54 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.269823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 811/1630] 10.222.0.72 () {54 vars in 912 bytes} [Thu Jun 16 14:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:07.272812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01203d-e49d-4f2b-9264-dc07b51ef634 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.272812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01203d-e49d-4f2b-9264-dc07b51ef634 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.272812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01203d-e49d-4f2b-9264-dc07b51ef634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.286565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 820/1631] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:07.288078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363dc2-674c-4c04-98c7-565cb5ad2f68 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.288378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363dc2-674c-4c04-98c7-565cb5ad2f68 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.288664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363dc2-674c-4c04-98c7-565cb5ad2f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.369295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-04363dc2-674c-4c04-98c7-565cb5ad2f68 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:07.371442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-04363dc2-674c-4c04-98c7-565cb5ad2f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:07.393321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1f01203d-e49d-4f2b-9264-dc07b51ef634 None None] MFA Rules not processed for user `8ee6194b42a540dfa40cd489cc7d404d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:07.395849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1f01203d-e49d-4f2b-9264-dc07b51ef634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:07.590581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 821/1632] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:07.597547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363dc2-674c-4c04-98c7-565cb5ad2f68 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:07.611378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 812/1633] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:07.612581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01203d-e49d-4f2b-9264-dc07b51ef634 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:07.756573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363dc2-674c-4c04-98c7-565cb5ad2f68 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:07.774504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:07.775666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.776990 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.777487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.813408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01203d-e49d-4f2b-9264-dc07b51ef634 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:07.818890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:07.819814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.820268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.820699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:07.840555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:07.842267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:07.850075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:07.852938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:07.858304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 813/1634] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:07.903366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 822/1635] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:07.963484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:07.975114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74a92b-3e90-4d12-a95c-fe34b1027e21 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:07.991672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:07.992360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:07.992645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:07.992957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:08.044764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:08.047031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:08.052743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:08.062489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 814/1636] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:07 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:08.408086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:08.512491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913960a-0a55-4f6c-a6ef-80d430bb5272 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:08.524194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:08.525355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:08.534323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:08.534769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] PATH_INFO: `/v3/users/9fda7395f88846e4a3d8b270fb1ee564` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:08.542944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fda7395f88846e4a3d8b270fb1ee564)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:08.544178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: WARNING py.warnings [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:57:08.544178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: warnings.warn(msg) Jun 16 14:57:08.544178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: Jun 16 14:57:08.546838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:08.705376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:08.755035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:08.772020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fda7395f88846e4a3d8b270fb1ee564'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:08.772020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fda7395f88846e4a3d8b270fb1ee564'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:08.772020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fda7395f88846e4a3d8b270fb1ee564'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:08.790125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d442fa-5a97-455a-bef4-e30ea52c5462 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:08.795631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fda7395f88846e4a3d8b270fb1ee564'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:08.819724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:08.838718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:08.839001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:08.840527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] PATH_INFO: `/v3/users/8036935d3cc547799faa68421de7ebcd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:08.843445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8036935d3cc547799faa68421de7ebcd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:08.856969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:08.907926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-6cd42147-53c5-4ed9-b502-39792e344c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fda7395f88846e4a3d8b270fb1ee564'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:09.014596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 823/1637] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:08 2022] DELETE /identity/v3/users/9fda7395f88846e4a3d8b270fb1ee564 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:09.021611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:09.022512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:09.027165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:09.111247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:09.112476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:09.117231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8036935d3cc547799faa68421de7ebcd'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:09.123591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8036935d3cc547799faa68421de7ebcd'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:09.142541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8036935d3cc547799faa68421de7ebcd'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:09.144315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8036935d3cc547799faa68421de7ebcd'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:09.158518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1e7ada54-bb7c-4e2e-b3d1-9bf651bda525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8036935d3cc547799faa68421de7ebcd'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:09.188550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 815/1638] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:08 2022] DELETE /identity/v3/users/8036935d3cc547799faa68421de7ebcd => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:09.191410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:09.191746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:09.192104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:09.220520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:09.222044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:09.313929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 824/1639] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:09.349057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 816/1640] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:09.352446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:09.354142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:09.377322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:09.379370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:09.602571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfc22d-2acb-424c-aca4-7c53d5827b68 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:09.637323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452d015-2004-45b7-bc2c-5d23620990bc None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:09.662019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:09.662802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:09.663190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:09.663804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:09.687361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:09.690775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:09.722172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 817/1641] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:57:09.755848 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:09.758830 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:09.764635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:09.765500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:09.796310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:09.805599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:09.831042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 825/1642] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:11.595155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:11.596726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:11.621395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e706c-649e-42cb-905f-84120958d206 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:11.635285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:11.636538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:11.636825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:11.637140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] PATH_INFO: `/v3/projects/5341374958f54cf29d9b203c0632cf7f` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:11.640354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5341374958f54cf29d9b203c0632cf7f)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:11.641681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: WARNING py.warnings [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 14:57:11.641681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: warnings.warn(msg) Jun 16 14:57:11.641681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: Jun 16 14:57:11.643132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:11.683566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:11.843156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5341374958f54cf29d9b203c0632cf7f', 'request_id': 'req-e02624d3-858b-48d0-b9d1-b5d1c6972f39'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:11.854401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5341374958f54cf29d9b203c0632cf7f', 'request_id': 'req-e02624d3-858b-48d0-b9d1-b5d1c6972f39'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:11.873285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5341374958f54cf29d9b203c0632cf7f', 'request_id': 'req-e02624d3-858b-48d0-b9d1-b5d1c6972f39'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:11.922895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] The token cache is being invalidate because project 5341374958f54cf29d9b203c0632cf7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:11.923360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:11.924277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:11.926771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 818/1643] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:11 2022] DELETE /identity/v3/projects/5341374958f54cf29d9b203c0632cf7f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:11.934135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:11.956654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c674eb-2b10-4e30-b895-d33409261c42 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:11.964247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:12.148479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:12.150687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:12.152309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:12.152797 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] PATH_INFO: `/v3/users/8ee6194b42a540dfa40cd489cc7d404d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:12.157297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ee6194b42a540dfa40cd489cc7d404d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:12.162525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:12.271764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ee6194b42a540dfa40cd489cc7d404d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.274614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ee6194b42a540dfa40cd489cc7d404d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.277986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ee6194b42a540dfa40cd489cc7d404d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.299530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ee6194b42a540dfa40cd489cc7d404d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.343465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ef20647b-d9b3-4ec9-b351-e193ba072b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ee6194b42a540dfa40cd489cc7d404d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.382086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 826/1644] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:11 2022] DELETE /identity/v3/users/8ee6194b42a540dfa40cd489cc7d404d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:12.480719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:12.483083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:12.517916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02624d3-858b-48d0-b9d1-b5d1c6972f39 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:12.664358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:12.665713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:12.666185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:12.666480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] PATH_INFO: `/v3/projects/420ebcc3dace46768bf717e019a47c2f` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:12.668653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=420ebcc3dace46768bf717e019a47c2f)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:12.671009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:12.798360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420ebcc3dace46768bf717e019a47c2f', 'request_id': 'req-acb2c89d-c0d5-4563-8732-c34a4a28b496'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.821574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420ebcc3dace46768bf717e019a47c2f', 'request_id': 'req-acb2c89d-c0d5-4563-8732-c34a4a28b496'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.848017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420ebcc3dace46768bf717e019a47c2f', 'request_id': 'req-acb2c89d-c0d5-4563-8732-c34a4a28b496'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.878501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] The token cache is being invalidate because project 420ebcc3dace46768bf717e019a47c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:12.878804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.879519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-acb2c89d-c0d5-4563-8732-c34a4a28b496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:12.881446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 819/1645] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:12 2022] DELETE /identity/v3/projects/420ebcc3dace46768bf717e019a47c2f => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:12.888491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ce40f-3027-4700-9f5d-011a11dbdd6c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:12.891132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ce40f-3027-4700-9f5d-011a11dbdd6c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:12.891764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ce40f-3027-4700-9f5d-011a11dbdd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:12.928707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-449ce40f-3027-4700-9f5d-011a11dbdd6c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:12.931164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-449ce40f-3027-4700-9f5d-011a11dbdd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:13.049425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 827/1646] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:13.059067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9cc0-ab0d-4ab4-862b-145762159dcb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:13.059557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9cc0-ab0d-4ab4-862b-145762159dcb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:13.060033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9cc0-ab0d-4ab4-862b-145762159dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:13.114568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c94f9cc0-ab0d-4ab4-862b-145762159dcb None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:13.116449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c94f9cc0-ab0d-4ab4-862b-145762159dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:13.144454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 820/1647] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:13.148901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ce40f-3027-4700-9f5d-011a11dbdd6c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:13.167829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ce40f-3027-4700-9f5d-011a11dbdd6c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:13.179698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:13.180202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:13.180880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:13.181439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:13.181831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:13.182158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:13.184356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:13.220776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 828/1648] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:13.225488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f9cc0-ab0d-4ab4-862b-145762159dcb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:13.238682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f9cc0-ab0d-4ab4-862b-145762159dcb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:13.256750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:13.257544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:13.257832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:13.258108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:13.258839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:13.261034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:13.332693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:13.371878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 821/1649] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:13 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:13.376784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:13.398872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67ed0d-b3d3-4170-aa31-c1fe3d78b48a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:13.420387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:13.640646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:13.641391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:13.641693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:13.641966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:13.645698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:13.650133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:13.707636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:13.711417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 829/1650] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:13 2022] POST /identity/v3/users => generated 285 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:13.736765 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d9843-b07e-4e4d-b96c-ac38221ffa16 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:13.776129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:13.778648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:13.779015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:13.779363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:13.780252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:13.784767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:13.847357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:13.880436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733d18d-cd52-4f68-9a3e-10e3b36a5a09 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:13.882945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 822/1651] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:13.914642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bda54b-8143-4058-aeb0-30642480dc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:13.927254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda54b-8143-4058-aeb0-30642480dc6f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:13.927254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda54b-8143-4058-aeb0-30642480dc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:13.927254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bda54b-8143-4058-aeb0-30642480dc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:13.927254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bda54b-8143-4058-aeb0-30642480dc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:13.927254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bda54b-8143-4058-aeb0-30642480dc6f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:13.970800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 830/1652] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:13.979184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:14.012240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc2fbb-e2e2-436b-9e36-e952c9d221a5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:14.074319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:14.081434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:14.082035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:14.082511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] PATH_INFO: `/v3/projects/d0dba4c446a94bda83f4acad3edd7c73/users/a81740dadbe14771b62fff2e1aa05f5e/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:14.138488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0dba4c446a94bda83f4acad3edd7c73, user_id=a81740dadbe14771b62fff2e1aa05f5e, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:14.143410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:14.220480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 823/1653] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:13 2022] PUT /identity/v3/projects/d0dba4c446a94bda83f4acad3edd7c73/users/a81740dadbe14771b62fff2e1aa05f5e/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:14.233454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279b85c-53e3-432a-9a7c-bf47d799406e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:14.234574 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279b85c-53e3-432a-9a7c-bf47d799406e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:14.235104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2279b85c-53e3-432a-9a7c-bf47d799406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:14.325756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2279b85c-53e3-432a-9a7c-bf47d799406e None None] MFA Rules not processed for user `a81740dadbe14771b62fff2e1aa05f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:14.330928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2279b85c-53e3-432a-9a7c-bf47d799406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:14.474119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 831/1654] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:14.479084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:14.574941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478218ed-244e-4c18-9ee3-e4880b03f9ac admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:14.582843 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:14.584251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:14.584624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:14.585072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:14.585989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:14.589659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:14.634749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 824/1655] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:14 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:14.639610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279b85c-53e3-432a-9a7c-bf47d799406e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:14.646746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:14.649689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:14.657492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2279b85c-53e3-432a-9a7c-bf47d799406e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:14.786338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:14.788493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:14.788979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:14.792579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:14.793822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:14.795809 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:14.821798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee927e-fc89-4490-89e4-7343464eddd3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:14.831082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 832/1656] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:14 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:14.832113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:14.834496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:14.836501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:14.836994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:14.837412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] PATH_INFO: `/v3/projects/91e0eaa896984bb8a84c069fadcb162e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:14.839369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e0eaa896984bb8a84c069fadcb162e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:14.842613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:14.876491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c847ec9-7178-4a6e-affc-cc90bc4324af admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:14.918100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:14.919016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:14.922044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:14.923879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:14.924877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:14.931098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:14.949945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:14.965901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e0eaa896984bb8a84c069fadcb162e', 'request_id': 'req-a1ecd744-0119-4938-bdeb-7690ddb9e920'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:14.967541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:14.970505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 833/1657] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:14.978284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:14.989475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e0eaa896984bb8a84c069fadcb162e', 'request_id': 'req-a1ecd744-0119-4938-bdeb-7690ddb9e920'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:15.003254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf3d33-8245-497d-9653-024479187594 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:15.007316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e0eaa896984bb8a84c069fadcb162e', 'request_id': 'req-a1ecd744-0119-4938-bdeb-7690ddb9e920'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:15.031160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] The token cache is being invalidate because project 91e0eaa896984bb8a84c069fadcb162e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:15.031457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:15.035934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1ecd744-0119-4938-bdeb-7690ddb9e920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:15.044302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 825/1658] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:14 2022] DELETE /identity/v3/projects/91e0eaa896984bb8a84c069fadcb162e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:15.050680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a94c6-014f-4379-80cc-53d7ab016030 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.051548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a94c6-014f-4379-80cc-53d7ab016030 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.051834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a94c6-014f-4379-80cc-53d7ab016030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.111712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-af8a94c6-014f-4379-80cc-53d7ab016030 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:15.113515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-af8a94c6-014f-4379-80cc-53d7ab016030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:15.185307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:15.186507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.186887 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.187297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] PATH_INFO: `/v3/projects/a9c9bfef31ea448b82ae2cf273fe251d/users/cb6eab5eebd44c85afb49027caec197f/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.211494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c9bfef31ea448b82ae2cf273fe251d, user_id=cb6eab5eebd44c85afb49027caec197f, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:15.215568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:15.264754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 834/1659] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:14 2022] PUT /identity/v3/projects/a9c9bfef31ea448b82ae2cf273fe251d/users/cb6eab5eebd44c85afb49027caec197f/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:15.266715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:15.268071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 826/1660] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:15.270092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:15.276513 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49a29-3404-4d92-8fef-18b8a0d80492 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.278213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49a29-3404-4d92-8fef-18b8a0d80492 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.278929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49a29-3404-4d92-8fef-18b8a0d80492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.321995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-57a49a29-3404-4d92-8fef-18b8a0d80492 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:15.322808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-57a49a29-3404-4d92-8fef-18b8a0d80492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:15.481453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257cd662-993e-4595-aa78-3341486762bd admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:15.506719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:15.507439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.507728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.508010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.508767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:15.510518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:15.552989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 835/1661] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:15.563392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 827/1662] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:15.563999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:15.565399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49a29-3404-4d92-8fef-18b8a0d80492 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:15.599483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49a29-3404-4d92-8fef-18b8a0d80492 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:15.619234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:15.620054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.620348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.620699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.621522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:15.621839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:15.623486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:15.639442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd959114-ab85-47c4-b28c-b1fc7c135388 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:15.669480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 828/1663] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:15.677912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:15.681395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e7680-0861-42b2-9952-d896b372cfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:15.682340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7680-0861-42b2-9952-d896b372cfc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.682666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7680-0861-42b2-9952-d896b372cfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.682949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7680-0861-42b2-9952-d896b372cfc4 admin admin] PATH_INFO: `/v3/projects/a9c9bfef31ea448b82ae2cf273fe251d/users/cb6eab5eebd44c85afb49027caec197f/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.698947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e7680-0861-42b2-9952-d896b372cfc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c9bfef31ea448b82ae2cf273fe251d, user_id=cb6eab5eebd44c85afb49027caec197f, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:15.700704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6228cec-3daf-4178-86d5-b74db7b7e63c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:15.708597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e7680-0861-42b2-9952-d896b372cfc4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:15.721231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:15.721928 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.722222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.722499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.723209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:15.725463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:15.756690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 836/1664] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:15 2022] PUT /identity/v3/projects/a9c9bfef31ea448b82ae2cf273fe251d/users/cb6eab5eebd44c85afb49027caec197f/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:15.758236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:15.762096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:15.762096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:15.796481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 829/1665] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:15 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:15.800672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:15.844471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] MFA Rules not processed for user `cb6eab5eebd44c85afb49027caec197f`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:15.850633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:15.934277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:15.999919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6547ad7-3254-4576-b9e4-5081ead137c0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:16.013446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:16.014158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.014446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.014727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.015439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:16.017442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:16.024388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:16.074187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 830/1666] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:15 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:16.079763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:16.085340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 837/1667] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:16.093625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:16.095644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:16.115415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd9af1-f085-4c3d-b089-ecdc03be5d9f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:16.128982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:16.129643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.129936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.130221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.130971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:16.132597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:16.176428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 831/1668] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:57:16.183289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:16.207629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cc244-f28d-4bc3-ae97-c1f29d2a7764 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:16.217784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:16.218915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.218915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.219069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.219814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:16.221498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:16.255054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 832/1669] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:57:16.257874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:16.283526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe33e54-da14-4007-977c-7fb27e678855 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:16.301752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9972b87-0a36-42ca-86aa-03629c24177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:16.302623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972b87-0a36-42ca-86aa-03629c24177e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.302909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972b87-0a36-42ca-86aa-03629c24177e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.303190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9972b87-0a36-42ca-86aa-03629c24177e admin admin] PATH_INFO: `/v3/projects/2c226754c0de49c491c2d16963863689/users/eb5996b5f1bf45778e17fa5ea4283303/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.305103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb34a11-5dfe-4e8b-a216-7d2f1cd24f4b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:16.327711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:16.328667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.329157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.329580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.343327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9972b87-0a36-42ca-86aa-03629c24177e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c226754c0de49c491c2d16963863689, user_id=eb5996b5f1bf45778e17fa5ea4283303, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:16.346145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9972b87-0a36-42ca-86aa-03629c24177e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:16.362464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:16.364230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:16.370977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 833/1670] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:16 2022] PUT /identity/v3/projects/2c226754c0de49c491c2d16963863689/users/eb5996b5f1bf45778e17fa5ea4283303/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:16.381685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670d27b-d57a-4a7e-a09a-7148b1d29864 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.381978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670d27b-d57a-4a7e-a09a-7148b1d29864 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.382265 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670d27b-d57a-4a7e-a09a-7148b1d29864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.408209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 838/1671] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:57:16.436684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9670d27b-d57a-4a7e-a09a-7148b1d29864 None None] MFA Rules not processed for user `eb5996b5f1bf45778e17fa5ea4283303`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:16.438021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9670d27b-d57a-4a7e-a09a-7148b1d29864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:16.541118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:16.541118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:16.562964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a535a-1394-416b-a9c1-dc3fe125667b service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:16.607667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 834/1672] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:16.609109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9345-1fae-4858-9b2d-e15e56a56e84 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.609408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9345-1fae-4858-9b2d-e15e56a56e84 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.610287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc9345-1fae-4858-9b2d-e15e56a56e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.655960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-20dc9345-1fae-4858-9b2d-e15e56a56e84 None None] MFA Rules not processed for user `ba40bcf173404e6c8b17d311b65bf780`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:16.657282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-20dc9345-1fae-4858-9b2d-e15e56a56e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:16.754465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:16.755158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:16.755452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:16.756385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:16.759088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:16.806523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 835/1673] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:16.808063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc9345-1fae-4858-9b2d-e15e56a56e84 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:16.832725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:16.917573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:16.921963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:16.946589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 839/1674] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:57:16.949442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:16.952353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:17.026930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc9345-1fae-4858-9b2d-e15e56a56e84 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:17.031909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:17.045218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:17.046038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.046391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.046732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:17.047515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:17.049838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:17.076085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea6eef-f87e-4204-be32-8d9dd7a8256e service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:17.083237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:17.084044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.084342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.084629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:17.096528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:17.098463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:17.112284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 840/1675] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:17.143263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 836/1676] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:16 2022] POST /identity/v3/projects => generated 373 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:17.147376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:17.164956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d141a7-57b2-409a-a6e0-b73f48ed7ed4 service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:17.310915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:17.311904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.313232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.315327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:17.316208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:17.325572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:17.363730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 841/1677] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:17 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:17.367045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:17.384134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12029c22-4db9-4570-b94b-cdf1ff39367a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:17.397273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:17.398036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.398243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.398517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:17.399463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:17.400949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:17.430088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 837/1678] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:17.434770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:17.446932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb5933-8c93-42d5-9727-5bbb2cd64c50 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:17.468759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:17.469855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.470251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.470658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] PATH_INFO: `/v3/projects/ae9442091e12479295a3d692a538749b/users/8021d5cccbaf4f06a8d0e7275bd7ec89/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:17.502016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9442091e12479295a3d692a538749b, user_id=8021d5cccbaf4f06a8d0e7275bd7ec89, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:17.507826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:17.554653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 842/1679] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:17 2022] PUT /identity/v3/projects/ae9442091e12479295a3d692a538749b/users/8021d5cccbaf4f06a8d0e7275bd7ec89/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:17.558881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:17.577635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356019a-e108-44dd-a8cc-e1bed704b9fa admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:17.751854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddf1b5-26cb-495b-a604-9a9036a906fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:17.751854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddf1b5-26cb-495b-a604-9a9036a906fb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.751854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddf1b5-26cb-495b-a604-9a9036a906fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.751854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddf1b5-26cb-495b-a604-9a9036a906fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:17.752507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddf1b5-26cb-495b-a604-9a9036a906fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:17.752507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddf1b5-26cb-495b-a604-9a9036a906fb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:17.801535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 838/1680] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:17.805981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:17.836787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a91d02-7476-40bc-aaad-a657d5b065cd admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:17.852872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:17.854939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.855231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.856749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] PATH_INFO: `/v3/projects/ae9442091e12479295a3d692a538749b/users/8021d5cccbaf4f06a8d0e7275bd7ec89/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:17.863175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9442091e12479295a3d692a538749b, user_id=8021d5cccbaf4f06a8d0e7275bd7ec89, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:17.877157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:17.917679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 843/1681] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:17 2022] PUT /identity/v3/projects/ae9442091e12479295a3d692a538749b/users/8021d5cccbaf4f06a8d0e7275bd7ec89/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:17.930255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201947b9-715c-470c-82f1-17d02f5eca6b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:17.930672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201947b9-715c-470c-82f1-17d02f5eca6b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:17.931017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201947b9-715c-470c-82f1-17d02f5eca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:18.038875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-201947b9-715c-470c-82f1-17d02f5eca6b None None] MFA Rules not processed for user `8021d5cccbaf4f06a8d0e7275bd7ec89`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:18.040931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-201947b9-715c-470c-82f1-17d02f5eca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:18.273860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 839/1682] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:18.285736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:18.431492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fe65c-b179-400a-babc-0f484fe84bf3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:18.453910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab463d0-4877-459c-921c-962578669094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:18.453910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab463d0-4877-459c-921c-962578669094 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:18.453910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab463d0-4877-459c-921c-962578669094 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:18.453910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab463d0-4877-459c-921c-962578669094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:18.474868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab463d0-4877-459c-921c-962578669094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:18.476398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab463d0-4877-459c-921c-962578669094 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:18.523287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 844/1683] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:18.737834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201947b9-715c-470c-82f1-17d02f5eca6b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:18.748761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:18.758085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201947b9-715c-470c-82f1-17d02f5eca6b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:18.774301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813678b-5d25-4a70-bcf3-f55b926e2835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:18.774999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813678b-5d25-4a70-bcf3-f55b926e2835 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:18.775292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813678b-5d25-4a70-bcf3-f55b926e2835 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:18.775573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813678b-5d25-4a70-bcf3-f55b926e2835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:18.960624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813678b-5d25-4a70-bcf3-f55b926e2835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:18.964918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813678b-5d25-4a70-bcf3-f55b926e2835 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:18.997797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 840/1684] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:57:21.903205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab463d0-4877-459c-921c-962578669094 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:21.906213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bab463d0-4877-459c-921c-962578669094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:22.053261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:22.165406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab463d0-4877-459c-921c-962578669094 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:22.180045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:22.181104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:22.181397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:22.182015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] PATH_INFO: `/v3/users/9ce13f0b56d84e88a611733039534715` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:22.185270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce13f0b56d84e88a611733039534715)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:22.193048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:22.323385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce13f0b56d84e88a611733039534715'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:22.323959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce13f0b56d84e88a611733039534715'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:22.325066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce13f0b56d84e88a611733039534715'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:22.342514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce13f0b56d84e88a611733039534715'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:22.377046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce13f0b56d84e88a611733039534715'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:22.405654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 845/1685] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:21 2022] DELETE /identity/v3/users/9ce13f0b56d84e88a611733039534715 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:22.407261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3e4d-63c4-4345-9bc9-c03d6c4258f3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:22.407751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3e4d-63c4-4345-9bc9-c03d6c4258f3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:22.407905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3e4d-63c4-4345-9bc9-c03d6c4258f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:22.431696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-156b3e4d-63c4-4345-9bc9-c03d6c4258f3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:22.433044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-156b3e4d-63c4-4345-9bc9-c03d6c4258f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:22.685050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 841/1686] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:22.754723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:22.756864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:22.970643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63110257-eec7-4d18-89aa-9677b0221394 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:22.978594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306924e2-4334-4908-bbd3-9537b376ab14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:22.981152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306924e2-4334-4908-bbd3-9537b376ab14 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:22.982863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306924e2-4334-4908-bbd3-9537b376ab14 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:22.983317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306924e2-4334-4908-bbd3-9537b376ab14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:23.022060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306924e2-4334-4908-bbd3-9537b376ab14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:23.025243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306924e2-4334-4908-bbd3-9537b376ab14 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:23.047659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:23.048004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:23.048375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:23.118397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 846/1687] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:23.148880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] MFA Rules not processed for user `1c246b5f70c54d678d827ea43d5c7a38`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:23.151699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:23.392635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 842/1688] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:23.394523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:23.394759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:23.395045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:23.406281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:23.408813 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:23.485538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6fa23-9578-48fc-838b-076e86df2176 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:23.485790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] MFA Rules not processed for user `1c246b5f70c54d678d827ea43d5c7a38`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:23.489891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:23.506688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:23.508071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:23.508992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:23.510190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] PATH_INFO: `/v3/users/a81740dadbe14771b62fff2e1aa05f5e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:23.514398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 847/1689] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:23.514714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81740dadbe14771b62fff2e1aa05f5e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:23.517370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:23.529645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:23.533089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:23.615846 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:23.715276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81740dadbe14771b62fff2e1aa05f5e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:23.718844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81740dadbe14771b62fff2e1aa05f5e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:23.775298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81740dadbe14771b62fff2e1aa05f5e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:23.777411 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81740dadbe14771b62fff2e1aa05f5e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:23.784661 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:23.795808 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9e46e9f9-2cf7-41c5-9a49-7758308c2951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81740dadbe14771b62fff2e1aa05f5e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:23.869093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 843/1690] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:23 2022] DELETE /identity/v3/users/a81740dadbe14771b62fff2e1aa05f5e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:23.873002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ed851-bf4e-4f0d-81ba-486a11486824 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:23.873295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ed851-bf4e-4f0d-81ba-486a11486824 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:23.873584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ed851-bf4e-4f0d-81ba-486a11486824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:23.898089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4aaab-932f-4517-8abe-4f6ae25e555f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:23.944165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-071ed851-bf4e-4f0d-81ba-486a11486824 None None] MFA Rules not processed for user `ba40bcf173404e6c8b17d311b65bf780`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:23.951940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-071ed851-bf4e-4f0d-81ba-486a11486824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:24.078216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 844/1691] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:24.080881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa76f26-6d29-4c2e-8841-227708cfb26c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:24.081224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa76f26-6d29-4c2e-8841-227708cfb26c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:24.081461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa76f26-6d29-4c2e-8841-227708cfb26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:24.084132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:24.084948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:24.085102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:24.085388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:24.087542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:24.156014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8aa76f26-6d29-4c2e-8841-227708cfb26c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:24.157300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8aa76f26-6d29-4c2e-8841-227708cfb26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:24.250852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:24.252502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:24.275694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 848/1692] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:24.289237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 845/1693] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:24.326615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:24.328302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:24.512925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810cf0d-0417-44d0-8120-e588d6e1c89d service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:24.534604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6cee4-708c-4be6-bed4-396d3477c0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:24.535255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6cee4-708c-4be6-bed4-396d3477c0bb service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:24.535553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6cee4-708c-4be6-bed4-396d3477c0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:24.535839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6cee4-708c-4be6-bed4-396d3477c0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:24.565392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6cee4-708c-4be6-bed4-396d3477c0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:24.566907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6cee4-708c-4be6-bed4-396d3477c0bb service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:24.589493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 849/1694] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:24.720028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4ff8-6b38-4de6-8930-6e563cf6845d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:24.720829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4ff8-6b38-4de6-8930-6e563cf6845d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:24.721274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df4ff8-6b38-4de6-8930-6e563cf6845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:24.776893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-30df4ff8-6b38-4de6-8930-6e563cf6845d None None] MFA Rules not processed for user `0b8be0ab53ae48af844bc281dee5f77c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:24.778427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-30df4ff8-6b38-4de6-8930-6e563cf6845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:24.868188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:24.932850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 846/1695] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:25.155407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.156292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.156729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.230469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] MFA Rules not processed for user `73a51a4e876e4910bbc5dd69f8a18058`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:25.232088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.266582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:25.381628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ff58-0061-48f7-8e8c-3abfe55b02cf None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.381922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ff58-0061-48f7-8e8c-3abfe55b02cf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.382204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935ff58-0061-48f7-8e8c-3abfe55b02cf None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.384048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 847/1696] 10.222.0.72 () {54 vars in 912 bytes} [Thu Jun 16 14:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:25.400801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd09ab-3a88-4a1d-82a3-53f9f5a5fb3f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.401201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd09ab-3a88-4a1d-82a3-53f9f5a5fb3f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.401913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd09ab-3a88-4a1d-82a3-53f9f5a5fb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.441940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 850/1697] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:25.443027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:25.444744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.467415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade130f4-9932-4372-8db0-47aa36391c12 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:25.473636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:25.475661 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.479854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.484798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] PATH_INFO: `/v3/projects/1107ea1ea92647f8a961cb49812b53bd` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.486560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1107ea1ea92647f8a961cb49812b53bd)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:25.489971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:25.500273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fedd09ab-3a88-4a1d-82a3-53f9f5a5fb3f None None] MFA Rules not processed for user `2fc5135327fe41409cc4ba407b004d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:25.502180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fedd09ab-3a88-4a1d-82a3-53f9f5a5fb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.519957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 848/1698] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:25.530839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.533617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.533617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.574963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] MFA Rules not processed for user `73a51a4e876e4910bbc5dd69f8a18058`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:25.576499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.600896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 849/1699] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:25.611569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:25.614451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1107ea1ea92647f8a961cb49812b53bd', 'request_id': 'req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:25.616315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.622835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1107ea1ea92647f8a961cb49812b53bd', 'request_id': 'req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:25.638041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1107ea1ea92647f8a961cb49812b53bd', 'request_id': 'req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:25.652080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0882fd9-16a9-4812-8507-bc3f628bf2d3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:25.661550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] The token cache is being invalidate because project 1107ea1ea92647f8a961cb49812b53bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:25.661963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:25.663518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21634ed9-1c30-4bec-b5e8-1a3c45bb6e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:25.682933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 851/1700] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:25 2022] DELETE /identity/v3/projects/1107ea1ea92647f8a961cb49812b53bd => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:25.688305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335735ae-bc27-40f8-aacf-879d3149da63 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.689449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335735ae-bc27-40f8-aacf-879d3149da63 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.690580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335735ae-bc27-40f8-aacf-879d3149da63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.741021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-335735ae-bc27-40f8-aacf-879d3149da63 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:25.744491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-335735ae-bc27-40f8-aacf-879d3149da63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.753949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8677-3173-4258-a9dd-2700052e27ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:25.755627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8677-3173-4258-a9dd-2700052e27ec service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.756122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8677-3173-4258-a9dd-2700052e27ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.756601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8677-3173-4258-a9dd-2700052e27ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.758935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7fdd8677-3173-4258-a9dd-2700052e27ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.786740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8677-3173-4258-a9dd-2700052e27ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:25.789132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8677-3173-4258-a9dd-2700052e27ec service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:25.803529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 850/1701] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:25.805275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768fe4b-6201-4794-9800-c6e746e05d0e None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.805603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768fe4b-6201-4794-9800-c6e746e05d0e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.806201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768fe4b-6201-4794-9800-c6e746e05d0e None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.816455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 851/1702] 10.222.0.72 () {54 vars in 912 bytes} [Thu Jun 16 14:57:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:25.824977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768fe4b-6201-4794-9800-c6e746e05d0e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:25.827364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8768fe4b-6201-4794-9800-c6e746e05d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.899467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 852/1703] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:25.901146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7c639-cf0a-4988-8876-43b4e847e172 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.901782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7c639-cf0a-4988-8876-43b4e847e172 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.902445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7c639-cf0a-4988-8876-43b4e847e172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:25.973173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c4a7c639-cf0a-4988-8876-43b4e847e172 None None] MFA Rules not processed for user `2fc5135327fe41409cc4ba407b004d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:25.974403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c4a7c639-cf0a-4988-8876-43b4e847e172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:25.995439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 853/1704] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:25.996867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:25.997122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:25.997410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.005713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768fe4b-6201-4794-9800-c6e746e05d0e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:26.015340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:26.016220 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.016507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.016781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] PATH_INFO: `/v3/projects/d0dba4c446a94bda83f4acad3edd7c73` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.022730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0dba4c446a94bda83f4acad3edd7c73)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:26.024923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:26.043938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:26.045404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:26.067852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 854/1705] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:26.073292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:26.076732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:26.099634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1efb79-def8-4f26-b8d7-ab747189a53d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:26.111658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:26.113633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.114365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.115169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.129477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0dba4c446a94bda83f4acad3edd7c73', 'request_id': 'req-897fd945-3587-44a0-b820-5d4fe9cd383c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.137177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:26.137486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:26.139912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:26.141316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0dba4c446a94bda83f4acad3edd7c73', 'request_id': 'req-897fd945-3587-44a0-b820-5d4fe9cd383c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.156074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 855/1706] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:26.157070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:26.159118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0dba4c446a94bda83f4acad3edd7c73', 'request_id': 'req-897fd945-3587-44a0-b820-5d4fe9cd383c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.181268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873580-1d74-4d44-9371-ae23fd9ad73d service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:26.188434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] The token cache is being invalidate because project d0dba4c446a94bda83f4acad3edd7c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:26.189538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.190896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:26.193536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.202127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 852/1707] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:25 2022] DELETE /identity/v3/projects/d0dba4c446a94bda83f4acad3edd7c73 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:26.206528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:26.211507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:26.335302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fd945-3587-44a0-b820-5d4fe9cd383c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:26.341647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:26.342410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:26.344404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.344404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.344404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] PATH_INFO: `/v3/users/cb6eab5eebd44c85afb49027caec197f` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.345881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb6eab5eebd44c85afb49027caec197f)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:26.348404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:26.349186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.349751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.350218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.352356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:26.355446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:26.362278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:26.388100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 856/1708] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:26.392963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:26.396461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:26.436599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7cafc-1376-4284-8160-0112dd9247c5 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:26.452049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb6eab5eebd44c85afb49027caec197f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.453052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb6eab5eebd44c85afb49027caec197f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.470977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb6eab5eebd44c85afb49027caec197f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.471990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb6eab5eebd44c85afb49027caec197f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.476324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1d9b1-d60a-4c64-b135-1df9505971c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:26.483112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1d9b1-d60a-4c64-b135-1df9505971c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.483711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb6eab5eebd44c85afb49027caec197f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:26.484108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1d9b1-d60a-4c64-b135-1df9505971c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.485640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1d9b1-d60a-4c64-b135-1df9505971c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.486771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1d9b1-d60a-4c64-b135-1df9505971c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:26.492607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1d9b1-d60a-4c64-b135-1df9505971c6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:26.510515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 853/1709] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:26 2022] DELETE /identity/v3/users/cb6eab5eebd44c85afb49027caec197f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:26.511976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:26.514444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:26.566857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 857/1710] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:26 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:26.569596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068c561-ba58-47c1-80a9-3dfea8255c6e None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.570232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068c561-ba58-47c1-80a9-3dfea8255c6e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.570617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068c561-ba58-47c1-80a9-3dfea8255c6e None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.574486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 858/1711] 10.222.0.72 () {54 vars in 960 bytes} [Thu Jun 16 14:57:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:26.575994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068c561-ba58-47c1-80a9-3dfea8255c6e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:26.578675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5068c561-ba58-47c1-80a9-3dfea8255c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:26.657053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:26.667393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4bce2-2c5f-4ec7-bca5-f6b845555227 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:26.762691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068c561-ba58-47c1-80a9-3dfea8255c6e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:26.768208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8ee50-685e-4c88-b385-c17196bba960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:26.769881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8ee50-685e-4c88-b385-c17196bba960 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.770192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8ee50-685e-4c88-b385-c17196bba960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.770474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8ee50-685e-4c88-b385-c17196bba960 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.771204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8ee50-685e-4c88-b385-c17196bba960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:26.773253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc8ee50-685e-4c88-b385-c17196bba960 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:26.834561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 859/1712] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:26 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:26.835978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.836273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.836556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.852324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:26.854550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:26.858458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:26.859528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:26.869181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:26.883054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:26.884737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:27.014634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.016784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.030202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 854/1713] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:27.031354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:27.046778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd900296-7296-4e65-9e41-2a19ef801be7 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:27.079209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 860/1714] 10.222.0.72 () {58 vars in 1057 bytes} [Thu Jun 16 14:57:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:27.080687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:27.087173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:27.090358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:27.092401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.093324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.095761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.098879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.104197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.151049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a974d0-6ee9-489f-a57e-7b93b2548355 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:27.164998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 855/1715] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:27.170116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:27.172826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:27.173961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.174449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.174938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.187471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:27.200604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:27.266485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f0d5b-926e-4489-983a-62d38f4be46c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:27.311390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:27.312006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.312287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.314100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.326996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:27.380240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.383506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.417363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 856/1716] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:27.418383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:27.439263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376da4c-92f7-4893-9961-b7b99dcac274 service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:27.445357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:27.446157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.446511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.446773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.448994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.452140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.489658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 857/1717] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:27.491327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:27.496182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:27.524694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.529730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.557398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 861/1718] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:27.559359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:27.576794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44dae7-b2a1-41f1-8649-7e90cf13e16e service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:27.580746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803c627-3da5-4d8f-9a4b-50c01478995b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:27.587705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:27.591185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.592114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.592548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] PATH_INFO: `/v3/projects/a7c6608ff8b043209e7f6c163aa5ad00/users/016e24ddbe0b4dc38cb5ffa0d5c73298/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.606633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe2ff2-3981-4da6-b5dd-aa35cd488659 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:27.607700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2ff2-3981-4da6-b5dd-aa35cd488659 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.608176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2ff2-3981-4da6-b5dd-aa35cd488659 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.608634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2ff2-3981-4da6-b5dd-aa35cd488659 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.633766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c6608ff8b043209e7f6c163aa5ad00, user_id=016e24ddbe0b4dc38cb5ffa0d5c73298, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.649834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe2ff2-3981-4da6-b5dd-aa35cd488659 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.650428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.654133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe2ff2-3981-4da6-b5dd-aa35cd488659 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.693064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 862/1719] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:27 2022] PUT /identity/v3/projects/a7c6608ff8b043209e7f6c163aa5ad00/users/016e24ddbe0b4dc38cb5ffa0d5c73298/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:27.706312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 858/1720] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:27.714274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:27.715966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.716561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.717065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.717348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:27.815638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678a57a-9b71-476e-85a0-89593eea039d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:27.830707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:27.833235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:27.834891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:27.835359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] PATH_INFO: `/v3/projects/a9c9bfef31ea448b82ae2cf273fe251d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:27.841344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c9bfef31ea448b82ae2cf273fe251d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:27.845695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:27.860120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] MFA Rules not processed for user `016e24ddbe0b4dc38cb5ffa0d5c73298`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:27.862609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:27.945219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c9bfef31ea448b82ae2cf273fe251d', 'request_id': 'req-0c34865e-a97a-4b57-a85e-614df63bac5f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:27.971624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c9bfef31ea448b82ae2cf273fe251d', 'request_id': 'req-0c34865e-a97a-4b57-a85e-614df63bac5f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:27.995472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c9bfef31ea448b82ae2cf273fe251d', 'request_id': 'req-0c34865e-a97a-4b57-a85e-614df63bac5f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:28.022224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] The token cache is being invalidate because project a9c9bfef31ea448b82ae2cf273fe251d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:28.022659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:28.023749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0c34865e-a97a-4b57-a85e-614df63bac5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:28.029201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:28.029382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 863/1721] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:27 2022] DELETE /identity/v3/projects/a9c9bfef31ea448b82ae2cf273fe251d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:28.036101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411644dd-e491-40fd-a98b-54d8c5a7274e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:28.036594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411644dd-e491-40fd-a98b-54d8c5a7274e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:28.037357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411644dd-e491-40fd-a98b-54d8c5a7274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:28.082126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-411644dd-e491-40fd-a98b-54d8c5a7274e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:28.083404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-411644dd-e491-40fd-a98b-54d8c5a7274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:28.098185 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:28.107294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 859/1722] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:28.109763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:28.114485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:28.166059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00284de6-b258-4ac3-b0e7-c7c57828c11d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:28.267989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 864/1723] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:28.284922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0944-1526-4e11-bbdf-611253870de8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:28.285222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0944-1526-4e11-bbdf-611253870de8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:28.285502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0944-1526-4e11-bbdf-611253870de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:28.339272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2c4b0944-1526-4e11-bbdf-611253870de8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:28.349388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2c4b0944-1526-4e11-bbdf-611253870de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:28.382668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 865/1724] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:28.388546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0944-1526-4e11-bbdf-611253870de8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:28.395033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:28.395733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:28.396022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:28.396302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:28.397055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:28.398976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:28.429000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0944-1526-4e11-bbdf-611253870de8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:28.470549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:28.471192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:28.471537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:28.477716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:28.478514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:28.479401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:28.485161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:28.569289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 860/1725] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:28 2022] POST /identity/v3/projects => generated 373 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:28.574824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:28.595407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 866/1726] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:57:28.600204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:28.618873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35726006-8870-45d5-b48b-eed04be8a7cd admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:28.894866 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:28.895642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:28.895936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:28.896214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:28.896969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:28.902856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:28.918820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306f5d8-0ddd-4ab2-8b26-d9d8aa56fd5c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:28.939304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:28.944336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:28.944952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:28.945373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:28.946304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:28.957489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:29.015220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 867/1727] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:28 2022] POST /identity/v3/projects => generated 387 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:29.020261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.045570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 861/1728] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:28 2022] POST /identity/v3/users => generated 293 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:29.049951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.085190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b701165-6f76-4a4e-9529-d090285858ac admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.251757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:29.273254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:29.275311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:29.276690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:29.278253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:29.279755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:29.283662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90da54f-9092-4316-b868-5a8dfd296939 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.285360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:29.314812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:29.317466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:29.317785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:29.318069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:29.318800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:29.324277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:29.324534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 862/1729] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:29.327663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.356555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce58f-228a-4105-83ec-d66e622a4cee admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.379573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:29.382183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:29.383083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:29.383476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] PATH_INFO: `/v3/projects/865a0ce0b5684242981a5fdfefb7f5ec/users/78d3000ef7dd4a3489f79a0a2d53e324/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:29.410283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 868/1730] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:29 2022] POST /identity/v3/users => generated 294 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:29.413926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.456486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b1f9a-f98b-4417-9c72-919fca968afc admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.468708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=865a0ce0b5684242981a5fdfefb7f5ec, user_id=78d3000ef7dd4a3489f79a0a2d53e324, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:29.478868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:29.499529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:29.507862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:29.508319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:29.508616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:29.513742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:29.515946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:29.518038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:29.522348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 863/1731] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:29 2022] PUT /identity/v3/projects/865a0ce0b5684242981a5fdfefb7f5ec/users/78d3000ef7dd4a3489f79a0a2d53e324/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:29.532590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.565444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 869/1732] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:29.569321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.609528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10120846-413a-45ce-a4ea-172fcc3818ee admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.798528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb97b29-921c-4ab4-85f5-6a9a91fa5c7a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.816259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:29.821450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:29.822462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:29.823428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:29.826126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:29.835375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:29.840798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:29.841537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:29.841990 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:29.842415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:29.848566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:29.854207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:29.886966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 864/1733] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:29.888382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.897000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:29.898322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 870/1734] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:29.905924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:29.936530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661909a-8478-4d2c-821e-776c25e91fde admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.953219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:29.958624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:29.959378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:29.961578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] PATH_INFO: `/v3/users/8b2bb951dc0b4065bb7d95e0c350b705` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:29.965516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a39cba-32ed-4dab-872a-d4e8b3d14b86 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:29.968909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2bb951dc0b4065bb7d95e0c350b705)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:29.976441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:30.004383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088be60-3c25-474f-b390-80219053604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:30.005570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088be60-3c25-474f-b390-80219053604d admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:30.006008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088be60-3c25-474f-b390-80219053604d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:30.006613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088be60-3c25-474f-b390-80219053604d admin admin] PATH_INFO: `/v3/projects/865a0ce0b5684242981a5fdfefb7f5ec/users/78d3000ef7dd4a3489f79a0a2d53e324/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:30.011368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088be60-3c25-474f-b390-80219053604d admin admin] RBAC: Authorizing `identity:create_grant(project_id=865a0ce0b5684242981a5fdfefb7f5ec, user_id=78d3000ef7dd4a3489f79a0a2d53e324, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:30.023749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088be60-3c25-474f-b390-80219053604d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:30.089877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 871/1735] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:29 2022] PUT /identity/v3/projects/865a0ce0b5684242981a5fdfefb7f5ec/users/78d3000ef7dd4a3489f79a0a2d53e324/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:30.093218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088be60-3c25-474f-b390-80219053604d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:30.210612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2bb951dc0b4065bb7d95e0c350b705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:30.213188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2bb951dc0b4065bb7d95e0c350b705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:30.249844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2bb951dc0b4065bb7d95e0c350b705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:30.252755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2bb951dc0b4065bb7d95e0c350b705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:30.278214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-146080cf-285f-4e74-9bdf-15dca557a549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2bb951dc0b4065bb7d95e0c350b705'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:30.386038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 865/1736] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:29 2022] DELETE /identity/v3/users/8b2bb951dc0b4065bb7d95e0c350b705 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:30.391950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5ffb-19fc-4c15-a5a0-0b350c6b7a85 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:30.405130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5ffb-19fc-4c15-a5a0-0b350c6b7a85 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:30.408446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b5ffb-19fc-4c15-a5a0-0b350c6b7a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:30.459193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088be60-3c25-474f-b390-80219053604d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:30.504684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-707b5ffb-19fc-4c15-a5a0-0b350c6b7a85 None None] MFA Rules not processed for user `73a51a4e876e4910bbc5dd69f8a18058`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:30.520295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-707b5ffb-19fc-4c15-a5a0-0b350c6b7a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:30.653422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:30.767769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 866/1737] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:30.769936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926023e1-3ab8-41f2-9322-bf302f61a971 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:30.771302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926023e1-3ab8-41f2-9322-bf302f61a971 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:30.771868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926023e1-3ab8-41f2-9322-bf302f61a971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:30.882891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-926023e1-3ab8-41f2-9322-bf302f61a971 None None] MFA Rules not processed for user `78d3000ef7dd4a3489f79a0a2d53e324`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:30.884884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-926023e1-3ab8-41f2-9322-bf302f61a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:30.913168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3231b83-ff5d-494d-9290-da48ae722b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:30.922845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3231b83-ff5d-494d-9290-da48ae722b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:30.934170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3231b83-ff5d-494d-9290-da48ae722b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:30.935119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3231b83-ff5d-494d-9290-da48ae722b50 admin admin] PATH_INFO: `/v3/projects/6bfb9d3cd9174574aa8dc21a80241303/users/4e868277ed8a4e11a9af481899d94ef9/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:31.024280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3231b83-ff5d-494d-9290-da48ae722b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfb9d3cd9174574aa8dc21a80241303, user_id=4e868277ed8a4e11a9af481899d94ef9, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:31.027487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3231b83-ff5d-494d-9290-da48ae722b50 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:31.070869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 872/1738] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:30 2022] PUT /identity/v3/projects/6bfb9d3cd9174574aa8dc21a80241303/users/4e868277ed8a4e11a9af481899d94ef9/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 980 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:31.073613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:31.074748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:31.076051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:31.083953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 867/1739] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:31.103868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:31.105930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:31.109417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:31.148915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:31.150862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:31.189120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:31.262613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] MFA Rules not processed for user `4e868277ed8a4e11a9af481899d94ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:31.267953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:31.411440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 873/1740] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:31.413334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:31.416029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:31.587912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 868/1741] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:31.587912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:31.589833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:31.653907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba1ff3-e771-435a-a5fc-a7c95255b560 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:31.665250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:31.665933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:31.666263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:31.668612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:31.776918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5798556-f4ca-4d3f-b669-cde2d43960d0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:31.790037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:31.790725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:31.791158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:31.792153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:31.825647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:31.830062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:31.848798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:31.879549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 869/1742] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:31.882073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:31.890539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:31.930364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:31.932689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:31.943512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004798c4-ec5e-4529-b0a4-1124902b103e service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:31.956981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 874/1743] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:31.967855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:31.968532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:31.970269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:31.970737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:31.972722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:31.979489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:32.092442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 870/1744] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:31 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:32.096854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:32.115312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3a499-8f6f-47d1-bc70-dea162bbb6c4 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:32.322332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:32.328248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:32.328759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:32.329193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:32.330140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:32.332723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:32.343748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:32.400033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 875/1745] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:32 2022] POST /identity/v3/users => generated 299 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:32.405411 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:32.424258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf2072-3d10-4112-ad49-8d632833868b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:32.435285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:32.437189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:32.439757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:32.440072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:32.440923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:32.442509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:32.494356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 871/1746] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:32.495384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:32.512705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131afc9-bf24-4e68-8376-068ed7539bb3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:32.525294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:32.528013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:32.529802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:32.530767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] PATH_INFO: `/v3/projects/367d1a668b864a7398c55166005820e3/users/fce398aad2e64da69477586602395912/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:32.559999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] RBAC: Authorizing `identity:create_grant(project_id=367d1a668b864a7398c55166005820e3, user_id=fce398aad2e64da69477586602395912, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:32.563851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:32.589260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 876/1747] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:32 2022] PUT /identity/v3/projects/367d1a668b864a7398c55166005820e3/users/fce398aad2e64da69477586602395912/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:32.593987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:32.612763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fc414-098b-44aa-b09f-cf757a3d8e54 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:32.747288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:32.748293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:32.748750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:32.749231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:32.750499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:32.753870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:32.780519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:32.783396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 872/1748] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:57:32.795473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ca299-9fb4-4402-93cb-5aefb9a3519c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:32.822375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5243b64-4f9f-4b97-b073-36d19f8f7cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:32.827623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5243b64-4f9f-4b97-b073-36d19f8f7cde admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:32.828944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5243b64-4f9f-4b97-b073-36d19f8f7cde admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:32.829491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5243b64-4f9f-4b97-b073-36d19f8f7cde admin admin] PATH_INFO: `/v3/projects/367d1a668b864a7398c55166005820e3/users/fce398aad2e64da69477586602395912/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:32.846969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5243b64-4f9f-4b97-b073-36d19f8f7cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=367d1a668b864a7398c55166005820e3, user_id=fce398aad2e64da69477586602395912, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:32.855511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:32.861107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:32.900260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5243b64-4f9f-4b97-b073-36d19f8f7cde admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:32.932644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99790f1-2dfc-4a92-8baa-078fac4c993e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:32.956245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 877/1749] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:32 2022] PUT /identity/v3/projects/367d1a668b864a7398c55166005820e3/users/fce398aad2e64da69477586602395912/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:32.976805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:32.977279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:32.977571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:33.031603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] MFA Rules not processed for user `fce398aad2e64da69477586602395912`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:33.032991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:33.113685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:33.115803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:33.116411 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:33.118092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] PATH_INFO: `/v3/users/016e24ddbe0b4dc38cb5ffa0d5c73298` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:33.123191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016e24ddbe0b4dc38cb5ffa0d5c73298)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:33.130626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:33.332697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 878/1750] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:33.341276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016e24ddbe0b4dc38cb5ffa0d5c73298'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:33.343952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016e24ddbe0b4dc38cb5ffa0d5c73298'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:33.350976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:33.352319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:33.374758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016e24ddbe0b4dc38cb5ffa0d5c73298'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:33.376387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016e24ddbe0b4dc38cb5ffa0d5c73298'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:33.409307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b96bd3f5-0591-4460-9c78-a31671677351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016e24ddbe0b4dc38cb5ffa0d5c73298'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:33.427505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:33.525121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0955e-82dc-4106-8311-722bb4ae5723 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:33.538189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:33.540320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:33.540624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:33.540945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:33.551058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 873/1751] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:32 2022] DELETE /identity/v3/users/016e24ddbe0b4dc38cb5ffa0d5c73298 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:33.551373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:33.557524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:33.558669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:33.558669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:33.602262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:33.605777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:33.607190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:33.607410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:33.653974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 879/1752] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:33.840857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:33.842531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:33.898025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 874/1753] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:57:33.903170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ca93b-0be9-4884-a474-79aecf47b1f3 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:33.940719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:33.941684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:33.941969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:33.942253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] PATH_INFO: `/v3/users/4e868277ed8a4e11a9af481899d94ef9` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:33.953884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e868277ed8a4e11a9af481899d94ef9)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:33.955954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:34.114890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:INSERT=5,keystone:SELECT=90,keystone:DELETE=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:34.134243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e868277ed8a4e11a9af481899d94ef9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:34.136717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e868277ed8a4e11a9af481899d94ef9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:34.138153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e868277ed8a4e11a9af481899d94ef9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:34.146487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:34.147937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:34.154450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e868277ed8a4e11a9af481899d94ef9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:34.211592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86372d9b-7d09-4173-b0da-bdc26a9bb6ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e868277ed8a4e11a9af481899d94ef9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:34.255538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 880/1754] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:33 2022] DELETE /identity/v3/users/4e868277ed8a4e11a9af481899d94ef9 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:34.268447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d32f5-329c-4326-bac6-1127973bcd90 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:34.271830 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d32f5-329c-4326-bac6-1127973bcd90 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:34.272394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d32f5-329c-4326-bac6-1127973bcd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:34.372199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3ae3b-24b3-4fc7-945c-a89c955399d3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:34.381372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-717d32f5-329c-4326-bac6-1127973bcd90 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:34.388406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-717d32f5-329c-4326-bac6-1127973bcd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:34.506959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 881/1755] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:34.508394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3aa01-7bb5-49c2-bffa-902631f3eda8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:34.508692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3aa01-7bb5-49c2-bffa-902631f3eda8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:34.509037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3aa01-7bb5-49c2-bffa-902631f3eda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:34.547681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:34.548574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:34.548887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:34.549172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:34.557759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-11f3aa01-7bb5-49c2-bffa-902631f3eda8 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:34.558227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:34.559668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-11f3aa01-7bb5-49c2-bffa-902631f3eda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:34.585845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:34.587030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:34.632787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 875/1756] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:34.636537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:34.639402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:34.666329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01741d8-ae39-49e6-a7c0-8b15b22daa6c service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:34.684770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:34.687385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:34.687385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:34.688382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:34.716746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:34.722817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:34.745980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 882/1757] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:57:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:34.753520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 876/1758] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:35.045906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:35.046603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:35.046698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:35.086148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] MFA Rules not processed for user `0b8be0ab53ae48af844bc281dee5f77c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:35.087331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:35.103277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:35.107223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:35.142175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eed2d6-6459-415b-8fc8-78259139096a service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:35.152598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:35.155841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:35.156718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:35.157252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] PATH_INFO: `/v3/projects/42bcd02068ee48e7b1d5a551e943e0a1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:35.159735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42bcd02068ee48e7b1d5a551e943e0a1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:35.162134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:35.231320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:35.258054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 883/1759] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:35.261721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:35.278283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:35.284426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:35.289051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42bcd02068ee48e7b1d5a551e943e0a1', 'request_id': 'req-c25f5165-5eca-4bca-a96b-137d89ad7bb6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:35.299153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42bcd02068ee48e7b1d5a551e943e0a1', 'request_id': 'req-c25f5165-5eca-4bca-a96b-137d89ad7bb6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:35.321664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42bcd02068ee48e7b1d5a551e943e0a1', 'request_id': 'req-c25f5165-5eca-4bca-a96b-137d89ad7bb6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:35.347477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] The token cache is being invalidate because project 42bcd02068ee48e7b1d5a551e943e0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:35.347775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:35.349619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c25f5165-5eca-4bca-a96b-137d89ad7bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:35.352960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 877/1760] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:35 2022] DELETE /identity/v3/projects/42bcd02068ee48e7b1d5a551e943e0a1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:35.359674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce4e9-af8a-4662-8a50-1cb5d42918d4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:35.361559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce4e9-af8a-4662-8a50-1cb5d42918d4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:35.362000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ce4e9-af8a-4662-8a50-1cb5d42918d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:35.445196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-936ce4e9-af8a-4662-8a50-1cb5d42918d4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:35.447333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-936ce4e9-af8a-4662-8a50-1cb5d42918d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:35.516222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde344f8-acb1-4db8-82a6-a5a266ee6885 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:35.626304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 878/1761] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:35.640998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68cdd-4240-47a6-83ff-c0b9bc47d7bf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:35.641668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68cdd-4240-47a6-83ff-c0b9bc47d7bf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:35.642359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68cdd-4240-47a6-83ff-c0b9bc47d7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:35.669431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b7d68cdd-4240-47a6-83ff-c0b9bc47d7bf None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:35.671767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b7d68cdd-4240-47a6-83ff-c0b9bc47d7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:35.697946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 879/1762] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:35.704952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d68cdd-4240-47a6-83ff-c0b9bc47d7bf None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:35.718681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:35.719361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:35.719649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:35.719936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:35.726002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d68cdd-4240-47a6-83ff-c0b9bc47d7bf None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:35.729238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:35.740882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:35.741598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:35.741908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:35.742182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:35.743095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:35.743227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:35.744956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:35.789355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 880/1763] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:57:35.790712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:35.807647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10e0f3-6644-4a23-a1f5-d924dd4f1bdb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:35.821741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:35.823635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:35.824138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:35.824631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:35.825697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:35.830069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:35.853935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:35.856041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:35.866853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 884/1764] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 14:57:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 14:57:35.872688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 881/1765] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:35 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:35.898239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:36.039918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa8d9d-a367-4f40-8740-48b26bca4691 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:36.048691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:36.049771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:36.050166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:36.050590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:36.051541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:36.054471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:36.082732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:36.102862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 882/1766] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:35 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:36.104113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:36.133123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f847dd8-1bd5-4066-8925-ce23755e92b5 service swift] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:36.155182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:36.155856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:36.156222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:36.156551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:36.157467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:36.159514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:36.208348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 885/1767] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:36.214510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:36.243060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e658c-f3d0-447b-b59d-d2f568011203 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:36.256578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779ed6d-467a-4df5-9e14-6d8350574912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:36.258940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779ed6d-467a-4df5-9e14-6d8350574912 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:36.259682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779ed6d-467a-4df5-9e14-6d8350574912 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:36.261205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779ed6d-467a-4df5-9e14-6d8350574912 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:36.262344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779ed6d-467a-4df5-9e14-6d8350574912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:36.267711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779ed6d-467a-4df5-9e14-6d8350574912 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:36.403969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 883/1768] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:36.408874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:36.434026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398b30e-4552-4dac-86fc-b82718b97f0b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:36.472954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:36.472954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:36.472954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:36.472954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] PATH_INFO: `/v3/projects/065f6d2ec26f4d55a4f7fad409dcfe63/users/162ce8e0595d4644ae09d8c430437efe/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:36.574745 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=065f6d2ec26f4d55a4f7fad409dcfe63, user_id=162ce8e0595d4644ae09d8c430437efe, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:36.578670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:36.634266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 886/1769] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:36 2022] PUT /identity/v3/projects/065f6d2ec26f4d55a4f7fad409dcfe63/users/162ce8e0595d4644ae09d8c430437efe/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:36.651100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df4efc-502c-40da-bec5-94b331c5d589 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:36.651886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df4efc-502c-40da-bec5-94b331c5d589 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:36.652044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df4efc-502c-40da-bec5-94b331c5d589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:36.747798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d4df4efc-502c-40da-bec5-94b331c5d589 None None] MFA Rules not processed for user `162ce8e0595d4644ae09d8c430437efe`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:36.751519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d4df4efc-502c-40da-bec5-94b331c5d589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:36.920130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 884/1770] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:36.923973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:36.941918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:37.035958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a3909-ce4f-4ee4-b541-9d6807039e1e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:37.041628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:37.042311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:37.042603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:37.042885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:37.043605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:37.045680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:37.101162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 887/1771] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:36 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:37.109311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df4efc-502c-40da-bec5-94b331c5d589 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:37.137306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df4efc-502c-40da-bec5-94b331c5d589 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:37.308942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:37.311267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:37.313062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:37.313620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:37.316507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:37.321178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:37.384204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 885/1772] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:37 2022] POST /identity/v3/users => generated 297 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:37.386402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:37.411309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d9a25-d641-49cb-a681-a753500a1ec4 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:37.423803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:37.436889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:37.436889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:37.440908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:37.442787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:37.454187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:37.526888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 888/1773] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:37.527947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:37.551563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e017f6-c1e7-420c-8615-7c7dfe0f6085 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:37.562094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:37.563418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:37.564785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:37.569339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:37.569686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:37.570156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] PATH_INFO: `/v3/projects/d9f909ea643b457d8951b0ec42e4157a/users/1037adace088479591e10aed64eeb069/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:37.603932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2ff00-2c4b-4e7d-9e71-531e466b6978 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:37.626689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:37.627865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:37.629063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:37.629577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] PATH_INFO: `/v3/projects/a7c6608ff8b043209e7f6c163aa5ad00` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:37.637679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c6608ff8b043209e7f6c163aa5ad00)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:37.640603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:37.644199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f909ea643b457d8951b0ec42e4157a, user_id=1037adace088479591e10aed64eeb069, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:37.670853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:37.722390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 886/1774] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:37 2022] PUT /identity/v3/projects/d9f909ea643b457d8951b0ec42e4157a/users/1037adace088479591e10aed64eeb069/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:37.727056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:37.813889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c6608ff8b043209e7f6c163aa5ad00', 'request_id': 'req-38accc1c-d546-44cb-84fd-c1835ef9b460'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:37.835618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c6608ff8b043209e7f6c163aa5ad00', 'request_id': 'req-38accc1c-d546-44cb-84fd-c1835ef9b460'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:37.854539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c6608ff8b043209e7f6c163aa5ad00', 'request_id': 'req-38accc1c-d546-44cb-84fd-c1835ef9b460'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:37.923141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] The token cache is being invalidate because project a7c6608ff8b043209e7f6c163aa5ad00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:37.931552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:37.938921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:37.955983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 889/1775] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:37 2022] DELETE /identity/v3/projects/a7c6608ff8b043209e7f6c163aa5ad00 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:37.958210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:37.968197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:38.003860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:38.140732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5d6a9-ad6e-4aee-87df-1df6be2332bb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:38.351464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38accc1c-d546-44cb-84fd-c1835ef9b460 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:38.373382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:38.374923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:38.375226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:38.375623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] PATH_INFO: `/v3/users/78d3000ef7dd4a3489f79a0a2d53e324` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:38.377120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] RBAC: Authorizing `identity:delete_user(user_id=78d3000ef7dd4a3489f79a0a2d53e324)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:38.379986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:38.406628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:38.408582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:38.409215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:38.409659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:38.410729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:38.411174 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:38.414981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:38.495204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 887/1776] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 768 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:38.501003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:38.504059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:38.536507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78d3000ef7dd4a3489f79a0a2d53e324'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:38.544018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78d3000ef7dd4a3489f79a0a2d53e324'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:38.547488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78d3000ef7dd4a3489f79a0a2d53e324'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:38.581666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78d3000ef7dd4a3489f79a0a2d53e324'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:38.581982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3546673-6f43-4c00-bb55-11a133375029 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:38.590045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb65c2-69bc-4ee7-9654-eaa857331a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:38.591512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb65c2-69bc-4ee7-9654-eaa857331a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:38.591796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb65c2-69bc-4ee7-9654-eaa857331a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:38.592083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb65c2-69bc-4ee7-9654-eaa857331a87 admin admin] PATH_INFO: `/v3/projects/d9f909ea643b457d8951b0ec42e4157a/users/1037adace088479591e10aed64eeb069/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:38.594520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb65c2-69bc-4ee7-9654-eaa857331a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f909ea643b457d8951b0ec42e4157a, user_id=1037adace088479591e10aed64eeb069, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:38.598424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb65c2-69bc-4ee7-9654-eaa857331a87 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:38.625602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a84d5388-8911-4c19-b8c2-fe336402439c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78d3000ef7dd4a3489f79a0a2d53e324'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:38.637943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 888/1777] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:38 2022] PUT /identity/v3/projects/d9f909ea643b457d8951b0ec42e4157a/users/1037adace088479591e10aed64eeb069/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:38.674719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 890/1778] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:37 2022] DELETE /identity/v3/users/78d3000ef7dd4a3489f79a0a2d53e324 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:38.689957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:38.690661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:38.692185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:38.794083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] MFA Rules not processed for user `1037adace088479591e10aed64eeb069`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:38.799633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:38.970632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89597-9f20-4f25-9239-84abe5f9befa None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:38.970906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89597-9f20-4f25-9239-84abe5f9befa None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:38.971181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89597-9f20-4f25-9239-84abe5f9befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:38.986669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 889/1779] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:39.014865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:39.016694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:39.032462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-6fe89597-9f20-4f25-9239-84abe5f9befa None None] MFA Rules not processed for user `2fc5135327fe41409cc4ba407b004d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:39.042021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6fe89597-9f20-4f25-9239-84abe5f9befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:39.262481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11cdd1-0d46-43b5-9876-02cb680bd2b7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:39.278131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:39.279108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:39.280578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:39.281653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:39.328670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:39.334618 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:39.363887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:39.385852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 890/1780] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:57:39.391179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 891/1781] 10.222.0.86 () {58 vars in 1015 bytes} [Thu Jun 16 14:57:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:39.565576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:39.567724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:39.600446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a95112-50a5-4a76-a6bf-c462e135912e service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:39.626358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:39.628400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:39.628400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:39.631379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:39.649732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89597-9f20-4f25-9239-84abe5f9befa None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:39.664397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:39.667774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:39.695905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89597-9f20-4f25-9239-84abe5f9befa None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:39.708299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:39.713767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:39.714690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:39.715238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:39.722757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:39.753551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 891/1782] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:39.770822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:39.773735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:39.804329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc21ee-1633-4f3a-aa9b-dbd73d6130b0 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:39.856049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:39.857222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:39.857677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:39.859636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:39.915889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:39.917672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:39.940675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 892/1783] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:39.941830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:39.967981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e93fb-62aa-4bf1-8780-b789efd9362e service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:39.978547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:39.979511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:39.979940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:39.980749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:39.998899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:40.001363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:40.001644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:40.003095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:40.022678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 893/1784] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:40.069986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 892/1785] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:41.945904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:41.948695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:42.039677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:42.042114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:42.208779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:42.235526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbeb1f-a9ee-4368-af99-4cf625f5d92d service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:42.261636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:42.262724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:42.263115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:42.263892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] PATH_INFO: `/v3/projects/6bfb9d3cd9174574aa8dc21a80241303` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:42.265102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfb9d3cd9174574aa8dc21a80241303)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:42.268899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:42.298318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64465e11-061d-4bc2-bb9c-d8307fdca327 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:42.327977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:42.330486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:42.332651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:42.334854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] PATH_INFO: `/v3/projects/865a0ce0b5684242981a5fdfefb7f5ec` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:42.337024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=865a0ce0b5684242981a5fdfefb7f5ec)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:42.340054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:42.520414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfb9d3cd9174574aa8dc21a80241303', 'request_id': 'req-37457d26-5342-471e-9f90-32500d15d3c5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.537649 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfb9d3cd9174574aa8dc21a80241303', 'request_id': 'req-37457d26-5342-471e-9f90-32500d15d3c5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.552707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfb9d3cd9174574aa8dc21a80241303', 'request_id': 'req-37457d26-5342-471e-9f90-32500d15d3c5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.555547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '865a0ce0b5684242981a5fdfefb7f5ec', 'request_id': 'req-749c3caa-e73a-4991-b362-6fe2b76e41f6'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.587700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '865a0ce0b5684242981a5fdfefb7f5ec', 'request_id': 'req-749c3caa-e73a-4991-b362-6fe2b76e41f6'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.607212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] The token cache is being invalidate because project 6bfb9d3cd9174574aa8dc21a80241303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:42.607605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.608447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.611823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '865a0ce0b5684242981a5fdfefb7f5ec', 'request_id': 'req-749c3caa-e73a-4991-b362-6fe2b76e41f6'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.618539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 894/1786] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:41 2022] DELETE /identity/v3/projects/6bfb9d3cd9174574aa8dc21a80241303 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:42.620212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:42.623579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:42.647733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] The token cache is being invalidate because project 865a0ce0b5684242981a5fdfefb7f5ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:42.652913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.657195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-749c3caa-e73a-4991-b362-6fe2b76e41f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:42.664498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 893/1787] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:42 2022] DELETE /identity/v3/projects/865a0ce0b5684242981a5fdfefb7f5ec => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:42.671285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08f545-8715-4e5b-9912-63575a11a1b2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:42.671604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08f545-8715-4e5b-9912-63575a11a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:42.671836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08f545-8715-4e5b-9912-63575a11a1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:42.717401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ca08f545-8715-4e5b-9912-63575a11a1b2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:42.720197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ca08f545-8715-4e5b-9912-63575a11a1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:42.780871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:42.824291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37457d26-5342-471e-9f90-32500d15d3c5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:42.929256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 894/1788] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:42.941426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ffea-09ee-4c7a-9097-12ae1012b449 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:42.941890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ffea-09ee-4c7a-9097-12ae1012b449 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:42.944263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ffea-09ee-4c7a-9097-12ae1012b449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:43.005849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-24b6ffea-09ee-4c7a-9097-12ae1012b449 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:43.013977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-24b6ffea-09ee-4c7a-9097-12ae1012b449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:43.030526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:43.033311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:43.034089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:43.039193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 895/1789] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:43.040171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] PATH_INFO: `/v3/users/fce398aad2e64da69477586602395912` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:43.043184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce398aad2e64da69477586602395912)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:43.043688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ffea-09ee-4c7a-9097-12ae1012b449 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:43.051202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:43.075201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ffea-09ee-4c7a-9097-12ae1012b449 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:43.084970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:43.085669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:43.085980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:43.086262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:43.086957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:43.092383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:43.096442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:43.198282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 896/1790] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:43.209189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:43.262095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e655b-36a4-4f4b-990f-4ebc1da23f6b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:43.298661 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:43.300364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce398aad2e64da69477586602395912'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:43.308582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce398aad2e64da69477586602395912'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:43.309201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:43.311235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:43.316528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:43.317958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:43.329508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:43.351397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce398aad2e64da69477586602395912'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:43.357093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce398aad2e64da69477586602395912'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:43.381383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c05f8b2d-f6ec-4e36-839b-081d7dc28adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce398aad2e64da69477586602395912'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:43.415373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 897/1791] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:43 2022] POST /identity/v3/projects => generated 401 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:43.420767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:43.434359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:43.451561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 895/1792] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:42 2022] DELETE /identity/v3/users/fce398aad2e64da69477586602395912 => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:43.512487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:43.662762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245c063-98d5-4b6e-9775-d59205ec19ea admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:43.682668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a89bf-e53f-4c9c-acf5-ec45b06c7c44 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:43.683107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a89bf-e53f-4c9c-acf5-ec45b06c7c44 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:43.683285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a89bf-e53f-4c9c-acf5-ec45b06c7c44 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:43.685816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 896/1793] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 14:57:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:43.720685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa6a88-c252-4130-aa58-a557b4732eff None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:43.721042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa6a88-c252-4130-aa58-a557b4732eff None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:43.721299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa6a88-c252-4130-aa58-a557b4732eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:43.814749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-29fa6a88-c252-4130-aa58-a557b4732eff None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:43.815623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-29fa6a88-c252-4130-aa58-a557b4732eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:43.922596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:43.924970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:43.926297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:43.927458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:43.929696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:43.935689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:44.047360 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 898/1794] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:43 2022] POST /identity/v3/users => generated 301 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:44.054972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:44.117722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd24303-0957-4253-a638-8614c1e372ce admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:44.158188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:44.158908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:44.159325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:44.159624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:44.160477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:44.171179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:44.234293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 899/1795] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:44.235321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:44.235682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 897/1796] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 14:57:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:44.287904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44f632-338b-4d53-8ebd-e4b803b3255f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:44.317076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e50799-eec9-46d0-b4eb-7d49db7c5708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:44.332947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e50799-eec9-46d0-b4eb-7d49db7c5708 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:44.332947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e50799-eec9-46d0-b4eb-7d49db7c5708 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:44.332947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e50799-eec9-46d0-b4eb-7d49db7c5708 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:44.332947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e50799-eec9-46d0-b4eb-7d49db7c5708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:44.332947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e50799-eec9-46d0-b4eb-7d49db7c5708 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:44.378850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 900/1797] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:44.385833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa6a88-c252-4130-aa58-a557b4732eff None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:44.414493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa6a88-c252-4130-aa58-a557b4732eff None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:44.432815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:44.437007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:44.438113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:44.438400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] PATH_INFO: `/v3/projects/b72ec9375c9d4b20938855135f8d8af1/users/2b1a575f48954fe782e8bf1d2630c1b3/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:44.452384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:44.501993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72ec9375c9d4b20938855135f8d8af1, user_id=2b1a575f48954fe782e8bf1d2630c1b3, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:44.509829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:44.548350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 898/1798] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:44 2022] PUT /identity/v3/projects/b72ec9375c9d4b20938855135f8d8af1/users/2b1a575f48954fe782e8bf1d2630c1b3/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:44.559679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18959cb3-9176-413e-af36-1c7f13e14049 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:44.560715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18959cb3-9176-413e-af36-1c7f13e14049 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:44.561769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18959cb3-9176-413e-af36-1c7f13e14049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:44.690969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-18959cb3-9176-413e-af36-1c7f13e14049 None None] MFA Rules not processed for user `2b1a575f48954fe782e8bf1d2630c1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:44.698087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-18959cb3-9176-413e-af36-1c7f13e14049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:44.965940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 901/1799] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:44.970521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:45.097820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8c9a5-09c3-4d16-9844-6b72c68ae95e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:45.114027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:45.115198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:45.115985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:45.116541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:45.117592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:45.120741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:45.159697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 899/1800] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:44 2022] POST /identity/v3/projects => generated 397 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:45.164011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18959cb3-9176-413e-af36-1c7f13e14049 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:45.187001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18959cb3-9176-413e-af36-1c7f13e14049 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:45.344440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:45.439552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:45.445018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:45.445613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:45.446114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:45.447137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:45.453969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:45.511510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 902/1801] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:45 2022] POST /identity/v3/users => generated 305 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:45.512871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:45.528070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00daa4a3-82d2-4cd9-9ba2-a666b9697715 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:45.535004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:45.535867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:45.536460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:45.536790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:45.537993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:45.539975 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:45.558522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 900/1802] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:57:45.562214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:45.579890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88169ae-dcd0-410f-96f1-6857934cec93 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:45.589394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:45.591095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:45.591273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:45.591643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] PATH_INFO: `/v3/projects/936c6d9ddfea421cafa8239cdd80d63a/users/968d190e6a1a4559865b70d1b35b19f3/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:45.619997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=936c6d9ddfea421cafa8239cdd80d63a, user_id=968d190e6a1a4559865b70d1b35b19f3, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:45.625093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:45.664164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 903/1803] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:45 2022] PUT /identity/v3/projects/936c6d9ddfea421cafa8239cdd80d63a/users/968d190e6a1a4559865b70d1b35b19f3/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:45.666453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:45.684585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c3fe7-d9ef-458c-86b5-0e1903a4a2c8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:45.800604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:45.803619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:45.841699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:45.842701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:45.843982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:45.846347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:45.847729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:45.849906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:45.907310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 901/1804] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:45.907310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:45.910274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:45.955090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a86c9d-72a6-47b6-b35e-5cc370ce28eb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:45.969924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:45.971173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:45.971558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:45.972305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] PATH_INFO: `/v3/projects/367d1a668b864a7398c55166005820e3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:45.974462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=367d1a668b864a7398c55166005820e3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:45.977432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:46.037258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7262a-0c15-4582-b575-18e18dcb56a4 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:46.071587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '367d1a668b864a7398c55166005820e3', 'request_id': 'req-318d44fc-688a-4ad4-b118-95e122956c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:46.073706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:46.084435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '367d1a668b864a7398c55166005820e3', 'request_id': 'req-318d44fc-688a-4ad4-b118-95e122956c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:46.100327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '367d1a668b864a7398c55166005820e3', 'request_id': 'req-318d44fc-688a-4ad4-b118-95e122956c6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:46.115559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] The token cache is being invalidate because project 367d1a668b864a7398c55166005820e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:46.116015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:46.117139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:46.124685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 902/1805] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:45 2022] DELETE /identity/v3/projects/367d1a668b864a7398c55166005820e3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:46.124685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:46.124685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:46.172688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6518c3-f14e-4312-9244-dee68a4d6ed1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:46.174121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6518c3-f14e-4312-9244-dee68a4d6ed1 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.174874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6518c3-f14e-4312-9244-dee68a4d6ed1 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.175320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6518c3-f14e-4312-9244-dee68a4d6ed1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.179090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5a6518c3-f14e-4312-9244-dee68a4d6ed1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:46.203083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6518c3-f14e-4312-9244-dee68a4d6ed1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:46.205574 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6518c3-f14e-4312-9244-dee68a4d6ed1 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:46.225395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d44fc-688a-4ad4-b118-95e122956c6d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:46.234109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 904/1806] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:46.235806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116be97-3b35-4c12-860c-f54e7eeebc1d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.236215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116be97-3b35-4c12-860c-f54e7eeebc1d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.236599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116be97-3b35-4c12-860c-f54e7eeebc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.240729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ddaaf-0ae1-440a-a376-8c7d4c2f350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:46.243098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ddaaf-0ae1-440a-a376-8c7d4c2f350c admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.243553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ddaaf-0ae1-440a-a376-8c7d4c2f350c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.244205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ddaaf-0ae1-440a-a376-8c7d4c2f350c admin admin] PATH_INFO: `/v3/projects/936c6d9ddfea421cafa8239cdd80d63a/users/968d190e6a1a4559865b70d1b35b19f3/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.246940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ddaaf-0ae1-440a-a376-8c7d4c2f350c admin admin] RBAC: Authorizing `identity:create_grant(project_id=936c6d9ddfea421cafa8239cdd80d63a, user_id=968d190e6a1a4559865b70d1b35b19f3, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:46.250097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ddaaf-0ae1-440a-a376-8c7d4c2f350c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:46.266733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 903/1807] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:46 2022] PUT /identity/v3/projects/936c6d9ddfea421cafa8239cdd80d63a/users/968d190e6a1a4559865b70d1b35b19f3/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:46.283030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0803693-6b81-4136-9ebc-c75081c24dc8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.283328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0803693-6b81-4136-9ebc-c75081c24dc8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.307160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0803693-6b81-4136-9ebc-c75081c24dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.326016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-3116be97-3b35-4c12-860c-f54e7eeebc1d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:46.329563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3116be97-3b35-4c12-860c-f54e7eeebc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:46.374295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:46.528874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 905/1808] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:46.532292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb26fb-608f-42c1-90dc-aeac297ed046 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.533494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb26fb-608f-42c1-90dc-aeac297ed046 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.533873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb26fb-608f-42c1-90dc-aeac297ed046 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.535712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 906/1809] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 14:57:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:57:46.537304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d22c-0183-45cf-8d16-53547e4a013d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.537669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d22c-0183-45cf-8d16-53547e4a013d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.538020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0d22c-0183-45cf-8d16-53547e4a013d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.578899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8bf0d22c-0183-45cf-8d16-53547e4a013d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:46.580401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8bf0d22c-0183-45cf-8d16-53547e4a013d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:46.597113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 907/1810] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:46.598472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0d22c-0183-45cf-8d16-53547e4a013d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:46.614963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0d22c-0183-45cf-8d16-53547e4a013d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:46.630165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64cc9f-aabd-4964-ad19-a3c67b40ac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:46.632515 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64cc9f-aabd-4964-ad19-a3c67b40ac5d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.633215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64cc9f-aabd-4964-ad19-a3c67b40ac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.633710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64cc9f-aabd-4964-ad19-a3c67b40ac5d admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.634710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64cc9f-aabd-4964-ad19-a3c67b40ac5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:46.636103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64cc9f-aabd-4964-ad19-a3c67b40ac5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:46.638095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64cc9f-aabd-4964-ad19-a3c67b40ac5d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:46.661340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b0803693-6b81-4136-9ebc-c75081c24dc8 None None] MFA Rules not processed for user `968d190e6a1a4559865b70d1b35b19f3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:46.665277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b0803693-6b81-4136-9ebc-c75081c24dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:46.665715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 908/1811] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:57:46.667767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.668193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.668687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.692652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:46.693928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:46.777154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 904/1812] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:46.778420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0803693-6b81-4136-9ebc-c75081c24dc8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:46.779036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 909/1813] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 14:57:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:46.785054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:46.786817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:46.820078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0803693-6b81-4136-9ebc-c75081c24dc8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:46.833353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:46.834028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.834685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.834954 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.835655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:46.838224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:46.865415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf6630-adb0-4df3-b1f3-5e883d2f3981 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:46.876208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 905/1814] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:46 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:46.880198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:46.882662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:46.883416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:46.883927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:46.884323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:46.897359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:46.899841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:46.920958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 910/1815] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:57:46.937196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:46.993537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b1edd-ce3b-4f1b-879c-722400e3dd17 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.001009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.001784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.002256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.002609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.003501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.006188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.037339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.043936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:47.061854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 906/1816] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:46 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:47.062850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.071607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0661a0-36d2-4d37-899f-5450376d86f9 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.082143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811e558-b066-4a81-88d5-25241e5020e6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.097491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.098183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.098463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.099216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.099992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.101301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.116665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 907/1817] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:47.116665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.120664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:47.138831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:47.204821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.205637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.206018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.206304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.219155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.220950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.236263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 911/1818] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:47.237916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.247283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6762be-b6e3-46e7-82c7-74275523d7fa admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.251000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0c8a8-4ab6-413a-8df6-b4ef20a3cb73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.251894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0c8a8-4ab6-413a-8df6-b4ef20a3cb73 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.252175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0c8a8-4ab6-413a-8df6-b4ef20a3cb73 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.252549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0c8a8-4ab6-413a-8df6-b4ef20a3cb73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.254126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f188718-e994-452e-9a24-1b7c911343c9 service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.254471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-23d0c8a8-4ab6-413a-8df6-b4ef20a3cb73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:47.264467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.265371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.265646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.265929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.266681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.268488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.298393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 912/1819] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:47.301829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.317199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3e4ed-df8f-4d7e-86e4-c9de12254f4d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.327663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.329096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.329585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.330375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] PATH_INFO: `/v3/projects/79586622768040479858b1311d391edd/users/b5aa586aa49943ed8a05f4af10a6ca8a/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.361647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=79586622768040479858b1311d391edd, user_id=b5aa586aa49943ed8a05f4af10a6ca8a, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.364559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.364677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0c8a8-4ab6-413a-8df6-b4ef20a3cb73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.366209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0c8a8-4ab6-413a-8df6-b4ef20a3cb73 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.378427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 908/1820] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:47.398505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 913/1821] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:47 2022] PUT /identity/v3/projects/79586622768040479858b1311d391edd/users/b5aa586aa49943ed8a05f4af10a6ca8a/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:57:47.408996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf5b87-dc26-4a4a-b8f6-ac3fe3c835ef None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.409287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf5b87-dc26-4a4a-b8f6-ac3fe3c835ef None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.409572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf5b87-dc26-4a4a-b8f6-ac3fe3c835ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.438670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-58cf5b87-dc26-4a4a-b8f6-ac3fe3c835ef None None] MFA Rules not processed for user `b5aa586aa49943ed8a05f4af10a6ca8a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:47.440589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-58cf5b87-dc26-4a4a-b8f6-ac3fe3c835ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:47.516723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:47.529334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 909/1822] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:47.529553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.657554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba45d9a-1de2-40ce-bb80-d6206a1aff7f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.673137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.674097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.678313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.678313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.678313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.683605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.718447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 914/1823] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:47 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:57:47.723666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf5b87-dc26-4a4a-b8f6-ac3fe3c835ef None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.775075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf5b87-dc26-4a4a-b8f6-ac3fe3c835ef None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.871927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.872746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.873155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.873504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.874340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.876507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.912009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 910/1824] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:47 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:47.915683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.927213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b0981-4212-436a-830a-165417bb7dae admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.933349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.934059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.934469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.934753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.935500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.937649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:47.954729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 915/1825] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:47.959216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:47.969995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbdc5e-e2b5-4632-918a-8fd02cb7bf80 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:47.975042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:47.976050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:47.976253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:47.976500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] PATH_INFO: `/v3/projects/e26d9693f1b447c68aa4550f2fdd8a10/users/4627f9cbcfc0412cabd70e0d64ab5cdb/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:47.993690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26d9693f1b447c68aa4550f2fdd8a10, user_id=4627f9cbcfc0412cabd70e0d64ab5cdb, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:47.996355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:48.009518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 911/1826] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:47 2022] PUT /identity/v3/projects/e26d9693f1b447c68aa4550f2fdd8a10/users/4627f9cbcfc0412cabd70e0d64ab5cdb/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:48.014394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:48.024641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369db2e-01bd-4505-8495-aff17b11d42b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:48.078683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:48.085596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5557502-3220-464d-98e7-093bb9433fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:48.086447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5557502-3220-464d-98e7-093bb9433fbd admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:48.086889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5557502-3220-464d-98e7-093bb9433fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:48.087414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5557502-3220-464d-98e7-093bb9433fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:48.089008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5557502-3220-464d-98e7-093bb9433fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:48.090447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5557502-3220-464d-98e7-093bb9433fbd admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:48.107487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 916/1827] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:48.108713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:48.120268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e82d8-6c72-4091-a341-5a4c02902d9d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:48.128047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:48.129011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:48.129296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:48.129574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] PATH_INFO: `/v3/projects/e26d9693f1b447c68aa4550f2fdd8a10/users/4627f9cbcfc0412cabd70e0d64ab5cdb/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:48.131834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26d9693f1b447c68aa4550f2fdd8a10, user_id=4627f9cbcfc0412cabd70e0d64ab5cdb, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:48.134513 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:48.147461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 912/1828] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:48 2022] PUT /identity/v3/projects/e26d9693f1b447c68aa4550f2fdd8a10/users/4627f9cbcfc0412cabd70e0d64ab5cdb/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:48.159012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d5ed8-0d1a-4882-8948-bc586ce4ffe1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:48.159302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d5ed8-0d1a-4882-8948-bc586ce4ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:48.159585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d5ed8-0d1a-4882-8948-bc586ce4ffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:48.187617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-eb9d5ed8-0d1a-4882-8948-bc586ce4ffe1 None None] MFA Rules not processed for user `4627f9cbcfc0412cabd70e0d64ab5cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:48.188877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-eb9d5ed8-0d1a-4882-8948-bc586ce4ffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:48.295881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 917/1829] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:48.313109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:48.395210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bece9e-7e6d-479d-b934-fc3c8ac19800 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:48.412760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd20b00-9844-4dc0-9478-fa2e9fb12415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:48.417159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd20b00-9844-4dc0-9478-fa2e9fb12415 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:48.417812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd20b00-9844-4dc0-9478-fa2e9fb12415 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:48.418307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd20b00-9844-4dc0-9478-fa2e9fb12415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:48.446064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd20b00-9844-4dc0-9478-fa2e9fb12415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:48.448098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd20b00-9844-4dc0-9478-fa2e9fb12415 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:48.467950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 913/1830] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:49.030858 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d5ed8-0d1a-4882-8948-bc586ce4ffe1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:49.095939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d5ed8-0d1a-4882-8948-bc586ce4ffe1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:49.100491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:49.101396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:49.101677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:49.101957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] PATH_INFO: `/v3/users/b5aa586aa49943ed8a05f4af10a6ca8a` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:49.103139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5aa586aa49943ed8a05f4af10a6ca8a)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:49.105164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:49.157397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5aa586aa49943ed8a05f4af10a6ca8a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.158100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5aa586aa49943ed8a05f4af10a6ca8a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.158725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5aa586aa49943ed8a05f4af10a6ca8a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.166061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5aa586aa49943ed8a05f4af10a6ca8a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.181944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5aa586aa49943ed8a05f4af10a6ca8a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.193472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:49.201085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 918/1831] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:49 2022] DELETE /identity/v3/users/b5aa586aa49943ed8a05f4af10a6ca8a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:49.206017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:49.206537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:49.206631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:49.224150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:49.225399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:49.277152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:49.291525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 914/1832] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:49.301106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:49.302662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:49.362850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff779a63-ceab-47e7-a00b-e6c98715edee admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:49.366875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:49.367543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:49.367833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:49.368146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:49.379450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:49.380882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:49.391808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 919/1833] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:49.772804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:49.775011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:49.793180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a57c4-6bd9-47ae-8b61-6776844b7415 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:49.812738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:49.817435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:49.817915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:49.818180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] PATH_INFO: `/v3/projects/79586622768040479858b1311d391edd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:49.825821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79586622768040479858b1311d391edd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:49.827801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:49.903987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79586622768040479858b1311d391edd', 'request_id': 'req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.909667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79586622768040479858b1311d391edd', 'request_id': 'req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.917095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79586622768040479858b1311d391edd', 'request_id': 'req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.929102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] The token cache is being invalidate because project 79586622768040479858b1311d391edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:49.929402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.930173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:49.936274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 915/1834] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:49 2022] DELETE /identity/v3/projects/79586622768040479858b1311d391edd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:49.938105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:49.960444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547beb2f-4898-45d3-b490-f34a6734c38d service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:50.029824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:50.030888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:50.031317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:50.031664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] PATH_INFO: `/v3/users/4627f9cbcfc0412cabd70e0d64ab5cdb` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:50.033112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4627f9cbcfc0412cabd70e0d64ab5cdb)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:50.035334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:50.100904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4627f9cbcfc0412cabd70e0d64ab5cdb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.100904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4627f9cbcfc0412cabd70e0d64ab5cdb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.100904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4627f9cbcfc0412cabd70e0d64ab5cdb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.115984 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4627f9cbcfc0412cabd70e0d64ab5cdb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.133277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51dd2c53-9eda-48c4-85ae-2c8100982a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4627f9cbcfc0412cabd70e0d64ab5cdb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.161217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 920/1835] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:57:49 2022] DELETE /identity/v3/users/4627f9cbcfc0412cabd70e0d64ab5cdb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:50.494936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:50.496706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:50.513216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bfdba-6b8c-46d9-b32e-7bfddf1d7c68 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:50.580886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:50.582145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:50.582145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:50.582339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] PATH_INFO: `/v3/projects/e26d9693f1b447c68aa4550f2fdd8a10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:50.583599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26d9693f1b447c68aa4550f2fdd8a10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:50.585572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:50.640560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26d9693f1b447c68aa4550f2fdd8a10', 'request_id': 'req-08b818e5-97c1-4060-a64f-986a00e8379f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.646387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26d9693f1b447c68aa4550f2fdd8a10', 'request_id': 'req-08b818e5-97c1-4060-a64f-986a00e8379f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.653990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26d9693f1b447c68aa4550f2fdd8a10', 'request_id': 'req-08b818e5-97c1-4060-a64f-986a00e8379f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.665328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] The token cache is being invalidate because project e26d9693f1b447c68aa4550f2fdd8a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:57:50.665615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.666276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-08b818e5-97c1-4060-a64f-986a00e8379f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:57:50.667840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 916/1836] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:57:50 2022] DELETE /identity/v3/projects/e26d9693f1b447c68aa4550f2fdd8a10 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:50.672984 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730767f-815a-4477-9044-2adad6c962de None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:50.673509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730767f-815a-4477-9044-2adad6c962de None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:50.673618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730767f-815a-4477-9044-2adad6c962de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:50.695341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-1730767f-815a-4477-9044-2adad6c962de None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:50.696564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1730767f-815a-4477-9044-2adad6c962de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:50.713958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:50.771151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 921/1837] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:50.780214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6e594-61e3-4248-9ab8-e78ae78603cd None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:50.780992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6e594-61e3-4248-9ab8-e78ae78603cd None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:50.781592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6e594-61e3-4248-9ab8-e78ae78603cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:50.796178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:50.803490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-12d6e594-61e3-4248-9ab8-e78ae78603cd None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:50.805917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-12d6e594-61e3-4248-9ab8-e78ae78603cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:50.818077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 917/1838] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:50.822483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730767f-815a-4477-9044-2adad6c962de None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:50.833007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730767f-815a-4477-9044-2adad6c962de None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:50.837883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:50.838833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:50.839125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:50.839432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:50.840174 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:57:50.840490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:50.842143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:50.860503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 922/1839] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:57:50.861558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6e594-61e3-4248-9ab8-e78ae78603cd None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:50.871872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6e594-61e3-4248-9ab8-e78ae78603cd None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:50.877831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:50.878518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:50.878793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:50.879064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:50.879781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:50.881726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:50.907909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 918/1840] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:57:50 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:50.912251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:50.922845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7196193-3c6f-40b1-8a73-fdb572c8275c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:50.984896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:50.985522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:50.985804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:50.986076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:50.986785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:50.989403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:51.017403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 923/1841] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:57:50 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:57:51.021522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:51.032421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ef5a4-6d5d-4734-a292-835e40bfa5f7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:51.037066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:51.037743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:51.038020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:51.038402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:51.039181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:51.041523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:51.052526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 919/1842] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:57:51.057163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:51.071157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116524-81d7-438a-90ec-bbfe2478ba31 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:51.080314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc957215-e1c9-4c9a-85fa-28a55040eb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:51.081034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc957215-e1c9-4c9a-85fa-28a55040eb30 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:51.081330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc957215-e1c9-4c9a-85fa-28a55040eb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:51.081611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc957215-e1c9-4c9a-85fa-28a55040eb30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:51.082352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc957215-e1c9-4c9a-85fa-28a55040eb30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:51.083970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc957215-e1c9-4c9a-85fa-28a55040eb30 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:51.095672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 924/1843] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:57:51.099602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:51.110232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ec2e6-e5a4-4481-9011-6e238138ba50 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:51.114918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:51.115798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:51.116077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:51.116365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] PATH_INFO: `/v3/projects/e15c04f0160548c4b1927d4f04464fb9/users/fafc3468763148f4a4fb3b3216156e97/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:51.131139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e15c04f0160548c4b1927d4f04464fb9, user_id=fafc3468763148f4a4fb3b3216156e97, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:51.133848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:51.146791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 920/1844] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:57:51 2022] PUT /identity/v3/projects/e15c04f0160548c4b1927d4f04464fb9/users/fafc3468763148f4a4fb3b3216156e97/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:57:51.157993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebe824-3bd2-4a7f-8dca-bde98dafb506 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:51.158290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebe824-3bd2-4a7f-8dca-bde98dafb506 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:51.158572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebe824-3bd2-4a7f-8dca-bde98dafb506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:51.193004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-deebe824-3bd2-4a7f-8dca-bde98dafb506 None None] MFA Rules not processed for user `fafc3468763148f4a4fb3b3216156e97`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:51.194236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-deebe824-3bd2-4a7f-8dca-bde98dafb506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:51.267585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 925/1845] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:51.280548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:51.282333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:51.342104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36787881-0c3d-4014-b233-20e72628f892 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:51.346473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:51.346634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:51.346860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:51.347137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:51.357247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:51.358755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:51.368313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 921/1846] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:57:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:57:52.531454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:52.531970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:52.532076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:52.552109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:57:52.553536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:52.569272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:57:52.625362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 926/1847] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:57:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:57:52.637625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:57:52.639041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:57:52.757845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182dd5e-a2ff-4f9f-8286-1d6acb45cf66 service glance] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:57:52.763011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231dfb6-6542-4ef8-af44-c42c4aa323ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:57:52.763769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231dfb6-6542-4ef8-af44-c42c4aa323ee service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:57:52.764009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231dfb6-6542-4ef8-af44-c42c4aa323ee service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:57:52.764235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231dfb6-6542-4ef8-af44-c42c4aa323ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:57:52.774320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231dfb6-6542-4ef8-af44-c42c4aa323ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:57:52.776463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231dfb6-6542-4ef8-af44-c42c4aa323ee service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:57:52.785901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 922/1848] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 14:57:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 14:58:02.617394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=18 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:02.782136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:06.715787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:06.717318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:06.786721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e364a2-e7bf-4c85-8e9c-3f8f6368ad7e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:06.792002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:06.792914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:06.793206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:06.793488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] PATH_INFO: `/v3/users/2b1a575f48954fe782e8bf1d2630c1b3` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:06.794721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1a575f48954fe782e8bf1d2630c1b3)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:06.796686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:06.856453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1a575f48954fe782e8bf1d2630c1b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:06.856453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1a575f48954fe782e8bf1d2630c1b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:06.856453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1a575f48954fe782e8bf1d2630c1b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:06.863057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1a575f48954fe782e8bf1d2630c1b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:06.877783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1a575f48954fe782e8bf1d2630c1b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:06.894942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 927/1849] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:06 2022] DELETE /identity/v3/users/2b1a575f48954fe782e8bf1d2630c1b3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:06.899516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:06.901337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:06.902531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:06.922691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:06.923927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:07.003805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 923/1850] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:07.015325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:07.016893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:07.017649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:07.018473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:07.033681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d65bed-3d30-4e5c-b2e5-3f313e5e598a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:07.038422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:07.039331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:07.039616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:07.039895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] PATH_INFO: `/v3/users/162ce8e0595d4644ae09d8c430437efe` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:07.041288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=162ce8e0595d4644ae09d8c430437efe)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:07.043260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:07.099809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162ce8e0595d4644ae09d8c430437efe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.101363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162ce8e0595d4644ae09d8c430437efe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.109611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162ce8e0595d4644ae09d8c430437efe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.110672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162ce8e0595d4644ae09d8c430437efe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.113676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e22b5a-f9bd-443d-9223-4d122ff847f5 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:07.119058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4493af-844c-46e0-8546-05d30ca0ee0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:07.120017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4493af-844c-46e0-8546-05d30ca0ee0b service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:07.120533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4493af-844c-46e0-8546-05d30ca0ee0b service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:07.121055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4493af-844c-46e0-8546-05d30ca0ee0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:07.122262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162ce8e0595d4644ae09d8c430437efe'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.124609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7b4493af-844c-46e0-8546-05d30ca0ee0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:07.139114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4493af-844c-46e0-8546-05d30ca0ee0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:07.140927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4493af-844c-46e0-8546-05d30ca0ee0b service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:07.152589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 928/1851] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:07.159165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 924/1852] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:07 2022] DELETE /identity/v3/users/162ce8e0595d4644ae09d8c430437efe => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:07.163905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:07.164204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:07.164490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:07.189357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:07.190666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:07.221826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:07.336976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 929/1853] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:07.346751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:07.348642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:07.437801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d5323-d589-46cd-9690-4a28822f2fce admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:07.442319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:07.442920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:07.443207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:07.443489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:07.457918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:07.459406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:07.479369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 925/1854] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:07.595763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:07.597360 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:07.617868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138c642-9f83-46ea-be88-3d8783c28b50 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:07.625281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:07.626170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:07.626456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:07.626865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] PATH_INFO: `/v3/projects/b72ec9375c9d4b20938855135f8d8af1` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:07.627987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72ec9375c9d4b20938855135f8d8af1)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:07.630530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:07.688205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72ec9375c9d4b20938855135f8d8af1', 'request_id': 'req-d11cfa8f-9811-4174-aaf8-85b8b269e046'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.691875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72ec9375c9d4b20938855135f8d8af1', 'request_id': 'req-d11cfa8f-9811-4174-aaf8-85b8b269e046'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.702919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72ec9375c9d4b20938855135f8d8af1', 'request_id': 'req-d11cfa8f-9811-4174-aaf8-85b8b269e046'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.739405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] The token cache is being invalidate because project b72ec9375c9d4b20938855135f8d8af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:07.740016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.740576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.742751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 930/1855] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:07 2022] DELETE /identity/v3/projects/b72ec9375c9d4b20938855135f8d8af1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:07.746977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:07.757510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb1a51-8f87-4e40-ae7c-684d154ce536 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:07.775503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:07.843009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:07.843831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:07.844122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:07.844540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] PATH_INFO: `/v3/users/968d190e6a1a4559865b70d1b35b19f3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:07.845889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] RBAC: Authorizing `identity:delete_user(user_id=968d190e6a1a4559865b70d1b35b19f3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:07.848881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:07.918971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968d190e6a1a4559865b70d1b35b19f3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.920276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968d190e6a1a4559865b70d1b35b19f3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.933327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968d190e6a1a4559865b70d1b35b19f3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.933961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968d190e6a1a4559865b70d1b35b19f3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.944259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968d190e6a1a4559865b70d1b35b19f3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:07.972307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 926/1856] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:07 2022] DELETE /identity/v3/users/968d190e6a1a4559865b70d1b35b19f3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:08.066492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:08.067990 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:08.085558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cfa8f-9811-4174-aaf8-85b8b269e046 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:08.170283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:08.171840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:08.172354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:08.172906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] PATH_INFO: `/v3/projects/065f6d2ec26f4d55a4f7fad409dcfe63` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:08.176000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=065f6d2ec26f4d55a4f7fad409dcfe63)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:08.180646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:08.255947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:08.257952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065f6d2ec26f4d55a4f7fad409dcfe63', 'request_id': 'req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.273989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065f6d2ec26f4d55a4f7fad409dcfe63', 'request_id': 'req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.284531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065f6d2ec26f4d55a4f7fad409dcfe63', 'request_id': 'req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.300912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] The token cache is being invalidate because project 065f6d2ec26f4d55a4f7fad409dcfe63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:08.301295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.302046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.304457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 931/1857] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:08 2022] DELETE /identity/v3/projects/065f6d2ec26f4d55a4f7fad409dcfe63 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:08.308923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:08.310640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:08.332238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780a1b4-93e0-42fd-896f-c9ea7a70007d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:08.411746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:08.413064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:08.413262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:08.413578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] PATH_INFO: `/v3/users/1037adace088479591e10aed64eeb069` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:08.416881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1037adace088479591e10aed64eeb069)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:08.418877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:08.469277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:08.470716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:08.499246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b3d67-9d7c-4c98-8945-9679bdf6df58 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:08.510571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:08.512056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:08.512287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:08.512710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] PATH_INFO: `/v3/projects/936c6d9ddfea421cafa8239cdd80d63a` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:08.514892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=936c6d9ddfea421cafa8239cdd80d63a)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:08.517855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:08.530063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:08.575827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1037adace088479591e10aed64eeb069'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.577630 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1037adace088479591e10aed64eeb069'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.591134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1037adace088479591e10aed64eeb069'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.591985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1037adace088479591e10aed64eeb069'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.602052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc222c6e-7dbf-4452-8aae-662a2f2423a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1037adace088479591e10aed64eeb069'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.629242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 927/1858] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:08 2022] DELETE /identity/v3/users/1037adace088479591e10aed64eeb069 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:08.637602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936c6d9ddfea421cafa8239cdd80d63a', 'request_id': 'req-c174a667-f693-4e6a-873e-65197a419aab'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.647133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936c6d9ddfea421cafa8239cdd80d63a', 'request_id': 'req-c174a667-f693-4e6a-873e-65197a419aab'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.674654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936c6d9ddfea421cafa8239cdd80d63a', 'request_id': 'req-c174a667-f693-4e6a-873e-65197a419aab'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.695387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] The token cache is being invalidate because project 936c6d9ddfea421cafa8239cdd80d63a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:08.696202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.697889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c174a667-f693-4e6a-873e-65197a419aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:08.700295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 932/1859] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:08 2022] DELETE /identity/v3/projects/936c6d9ddfea421cafa8239cdd80d63a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:08.706294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b11ef-fbca-45c5-927e-7a7217d8e0a9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:08.706631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b11ef-fbca-45c5-927e-7a7217d8e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:08.707125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b11ef-fbca-45c5-927e-7a7217d8e0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:08.741765 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1e1b11ef-fbca-45c5-927e-7a7217d8e0a9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:08.743031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1e1b11ef-fbca-45c5-927e-7a7217d8e0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:08.936476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 928/1860] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:08.945843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ebae-eb93-4ecc-af58-02daf7e38355 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:08.946878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ebae-eb93-4ecc-af58-02daf7e38355 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:08.947365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ebae-eb93-4ecc-af58-02daf7e38355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:08.982341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-6bb0ebae-eb93-4ecc-af58-02daf7e38355 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:08.985473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6bb0ebae-eb93-4ecc-af58-02daf7e38355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:09.003628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 933/1861] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:09.008246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b11ef-fbca-45c5-927e-7a7217d8e0a9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.029998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b11ef-fbca-45c5-927e-7a7217d8e0a9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.036112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.037328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.037719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.038101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.039026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:09.039445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.043572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.063878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 929/1862] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:58:09.071541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ebae-eb93-4ecc-af58-02daf7e38355 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.111519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ebae-eb93-4ecc-af58-02daf7e38355 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.124773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.126258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.126861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.127771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.129658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.133782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.180432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 934/1863] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:09 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:09.182728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.199741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db146c10-98f8-4dae-adce-821ff9af6a11 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.315955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.316491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.317091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.317352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.318077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.320163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.345699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 930/1864] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:09 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:09.350718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.365192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8e260-3f09-4cff-bcab-bc13dc6c00d6 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.371336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.377191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.377191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.377191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.377191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.377191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.388174 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 935/1865] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:09.392937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.405197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747477c-c0f7-45d6-b3cf-8ace71617e80 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.422771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.424120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.426717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.428129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.429137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:09.430918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.432387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.442909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.451686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a17afd-22b1-4186-84cd-f6a55ed1ea9b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.461015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 931/1866] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:09.470061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.470777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.472097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.472395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.474078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] PATH_INFO: `/v3/projects/d9f909ea643b457d8951b0ec42e4157a` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.476996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f909ea643b457d8951b0ec42e4157a)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.482045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.497473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae49da66-6004-4f5d-a15e-3966479ac829 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.507404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304851f8-d473-4893-893f-a227560a5532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.509611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304851f8-d473-4893-893f-a227560a5532 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.509915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304851f8-d473-4893-893f-a227560a5532 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.510199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304851f8-d473-4893-893f-a227560a5532 admin admin] PATH_INFO: `/v3/projects/6086f7a800614e4ba1c1139c604cab0c/users/219721568989442d87e6b79a6cf02a30/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.529005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304851f8-d473-4893-893f-a227560a5532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6086f7a800614e4ba1c1139c604cab0c, user_id=219721568989442d87e6b79a6cf02a30, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.531987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304851f8-d473-4893-893f-a227560a5532 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.535412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:09.549984 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f909ea643b457d8951b0ec42e4157a', 'request_id': 'req-37d453b0-7c25-40fb-ba5c-f86e9e28c225'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:09.555474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 932/1867] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:09 2022] PUT /identity/v3/projects/6086f7a800614e4ba1c1139c604cab0c/users/219721568989442d87e6b79a6cf02a30/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:09.568030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f909ea643b457d8951b0ec42e4157a', 'request_id': 'req-37d453b0-7c25-40fb-ba5c-f86e9e28c225'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:09.576614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.577109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.577495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.587215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f909ea643b457d8951b0ec42e4157a', 'request_id': 'req-37d453b0-7c25-40fb-ba5c-f86e9e28c225'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:09.628443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] The token cache is being invalidate because project d9f909ea643b457d8951b0ec42e4157a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:09.628746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:09.630725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-37d453b0-7c25-40fb-ba5c-f86e9e28c225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:09.635535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 936/1868] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:09 2022] DELETE /identity/v3/projects/d9f909ea643b457d8951b0ec42e4157a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:09.641035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba34a-be3b-4eb0-8da9-29de6f583505 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.641854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba34a-be3b-4eb0-8da9-29de6f583505 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.645333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] MFA Rules not processed for user `219721568989442d87e6b79a6cf02a30`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:09.647268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:09.657232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ba34a-be3b-4eb0-8da9-29de6f583505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.673985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:09.738888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f46ba34a-be3b-4eb0-8da9-29de6f583505 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:09.740377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f46ba34a-be3b-4eb0-8da9-29de6f583505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:09.760925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 933/1869] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:09.769889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.773128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:09.828065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 937/1870] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:09.835770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5dac-1bd9-4e8d-ae7c-1bf4976aa33e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.835991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5dac-1bd9-4e8d-ae7c-1bf4976aa33e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.836099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e5dac-1bd9-4e8d-ae7c-1bf4976aa33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.858765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-755e5dac-1bd9-4e8d-ae7c-1bf4976aa33e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:09.860035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-755e5dac-1bd9-4e8d-ae7c-1bf4976aa33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:09.861376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c99fde-6e0c-4ec7-b3e7-c1d31dfa6b36 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.866000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.866938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.867454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.867861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.872883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 938/1871] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:09.873869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e5dac-1bd9-4e8d-ae7c-1bf4976aa33e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.882105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.883883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e5dac-1bd9-4e8d-ae7c-1bf4976aa33e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.884034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.893207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.893810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.894118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.894547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.895340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:09.895843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.897547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:09.899266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 934/1872] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:09.919617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 939/1873] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:58:09.934467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.936034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:09.937596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:09.963928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94512c3d-acf1-4a5b-a731-a91203753674 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.980654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe027c-9a0e-40aa-9985-5ba486f2acb2 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:09.983119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:09.984384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:09.984676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:09.985181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] PATH_INFO: `/v3/users/219721568989442d87e6b79a6cf02a30` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:09.986712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219721568989442d87e6b79a6cf02a30)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:09.988981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:10.012579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:10.014095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.014239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.014477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.015192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:10.029450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:10.054029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219721568989442d87e6b79a6cf02a30'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:10.054743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219721568989442d87e6b79a6cf02a30'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:10.055469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219721568989442d87e6b79a6cf02a30'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:10.065707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219721568989442d87e6b79a6cf02a30'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:10.085525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0a57f1f5-cc50-4681-bb72-fa879a48c190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219721568989442d87e6b79a6cf02a30'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:10.097903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 935/1874] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:09 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:10.101238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:10.116263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:10.116347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 940/1875] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:09 2022] DELETE /identity/v3/users/219721568989442d87e6b79a6cf02a30 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:10.121965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f159-e94a-4635-b356-5971157938d2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.122432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f159-e94a-4635-b356-5971157938d2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.122636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74f159-e94a-4635-b356-5971157938d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.178340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-da74f159-e94a-4635-b356-5971157938d2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:10.180101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-da74f159-e94a-4635-b356-5971157938d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:10.281383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:10.297609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f757bb-d24e-4384-a506-35ffa11d1e9c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:10.304309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 941/1876] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:10.313872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74f159-e94a-4635-b356-5971157938d2 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:10.314946 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:10.315855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.316051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-da74f159-e94a-4635-b356-5971157938d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:10.316324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.316701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.319194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:10.322170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:10.358214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:10.388762 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 936/1877] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:10 2022] POST /identity/v3/users => generated 294 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:10.390313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:10.417211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a9111-93da-4ced-8a34-223bd24bd8b7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:10.428644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c62c3-9156-4158-b9a8-2b6d3606b4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:10.430958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c62c3-9156-4158-b9a8-2b6d3606b4dd admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.432744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c62c3-9156-4158-b9a8-2b6d3606b4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.435331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c62c3-9156-4158-b9a8-2b6d3606b4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.436926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c62c3-9156-4158-b9a8-2b6d3606b4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:10.438977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c62c3-9156-4158-b9a8-2b6d3606b4dd admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:10.477832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 937/1878] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:10.487763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc44ce4-28a4-462c-9419-667b81cfa2e2 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.488304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc44ce4-28a4-462c-9419-667b81cfa2e2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.488676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc44ce4-28a4-462c-9419-667b81cfa2e2 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.493780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 938/1879] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 14:58:10 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:58:10.494160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc44ce4-28a4-462c-9419-667b81cfa2e2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:10.509390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74f159-e94a-4635-b356-5971157938d2 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:10.523409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2cada-5d2d-4556-8039-a7067ea9387a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:10.525119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2cada-5d2d-4556-8039-a7067ea9387a service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.527634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2cada-5d2d-4556-8039-a7067ea9387a service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.528372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2cada-5d2d-4556-8039-a7067ea9387a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.531282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc44ce4-28a4-462c-9419-667b81cfa2e2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:10.544858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:10.545614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.545930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.546206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.549480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:10.552544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:10.556092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2cada-5d2d-4556-8039-a7067ea9387a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:10.559179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2cada-5d2d-4556-8039-a7067ea9387a service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:10.589521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 939/1880] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:10.591886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 942/1881] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:10.595914 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:10.628143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8bb09-76f9-42b3-b968-7a388d6bc992 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:10.666244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1f88c-9b5e-4174-b36d-ecbbac225093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:10.669992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1f88c-9b5e-4174-b36d-ecbbac225093 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.670311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1f88c-9b5e-4174-b36d-ecbbac225093 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.670558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1f88c-9b5e-4174-b36d-ecbbac225093 admin admin] PATH_INFO: `/v3/projects/758d689409144c7b9e2a2fc6600cc297/users/9dd36f8bebb64863b6804b88cbd2f785/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.705944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1f88c-9b5e-4174-b36d-ecbbac225093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758d689409144c7b9e2a2fc6600cc297, user_id=9dd36f8bebb64863b6804b88cbd2f785, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:10.711243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1f88c-9b5e-4174-b36d-ecbbac225093 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:10.731214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 940/1882] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:10 2022] PUT /identity/v3/projects/758d689409144c7b9e2a2fc6600cc297/users/9dd36f8bebb64863b6804b88cbd2f785/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:10.742170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4219718-72b9-47c3-8a17-5fc5ed2b4280 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.742406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4219718-72b9-47c3-8a17-5fc5ed2b4280 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.742633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4219718-72b9-47c3-8a17-5fc5ed2b4280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.771405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4df28-f830-4259-8a6e-3ad7ffe9dab8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:10.771780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4df28-f830-4259-8a6e-3ad7ffe9dab8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:10.775309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4df28-f830-4259-8a6e-3ad7ffe9dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:10.796138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-d4219718-72b9-47c3-8a17-5fc5ed2b4280 None None] MFA Rules not processed for user `9dd36f8bebb64863b6804b88cbd2f785`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:10.797571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d4219718-72b9-47c3-8a17-5fc5ed2b4280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:10.819019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-51c4df28-f830-4259-8a6e-3ad7ffe9dab8 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:10.822555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-51c4df28-f830-4259-8a6e-3ad7ffe9dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:10.924024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 943/1883] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:10.928223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4219718-72b9-47c3-8a17-5fc5ed2b4280 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:10.933204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 941/1884] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:11.033535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4219718-72b9-47c3-8a17-5fc5ed2b4280 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.041355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.041355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.041355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.041355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.041804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.042859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.074782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 944/1885] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:10 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:11.078509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4df28-f830-4259-8a6e-3ad7ffe9dab8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.111194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4df28-f830-4259-8a6e-3ad7ffe9dab8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.193042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.195668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:11.235288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:11.278523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.279493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.279782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.280060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.280805 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.282819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.325980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 942/1886] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:11 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:11.334484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.349539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1b8da-3346-4002-ba49-ab37ef6e817e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.357226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.358698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.359168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.360091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.361454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.364425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.386875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 943/1887] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:58:11.392952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.408057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c025c08-a067-4c53-8a28-658f074666cb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.418992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1254cc-a7df-4e39-8599-75f3bdd07934 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.421821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.422846 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.424775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.425289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] PATH_INFO: `/v3/projects/bf69b3016b1d44f59eabb193013b685b/users/98e93e7684414365b10306267721c229/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.427383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.428057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.428347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.428627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.439688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.441409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.467878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf69b3016b1d44f59eabb193013b685b, user_id=98e93e7684414365b10306267721c229, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.468819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 945/1888] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:11.470011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.473379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.474380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:11.477377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:11.495385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 944/1889] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:11 2022] PUT /identity/v3/projects/bf69b3016b1d44f59eabb193013b685b/users/98e93e7684414365b10306267721c229/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:11.499280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.508604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49512b-10e6-4408-9c3a-2001fc5c7c00 service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.580025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b6088-dfdb-40b0-9542-cc9b052e6d9e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.582500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.583632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.583924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.584290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] PATH_INFO: `/v3/projects/6086f7a800614e4ba1c1139c604cab0c` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.585605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6086f7a800614e4ba1c1139c604cab0c)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.586280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.587169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.587415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.587700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.588032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.588561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.590228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.604580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 945/1890] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:11.609097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.620183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fd71-00e7-4964-ade5-d3f5ed608e4e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.625543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c1d8d-9b3d-4f73-9105-8854631925e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.626479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c1d8d-9b3d-4f73-9105-8854631925e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.626800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c1d8d-9b3d-4f73-9105-8854631925e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.627088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c1d8d-9b3d-4f73-9105-8854631925e1 admin admin] PATH_INFO: `/v3/projects/bf69b3016b1d44f59eabb193013b685b/users/98e93e7684414365b10306267721c229/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.629285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c1d8d-9b3d-4f73-9105-8854631925e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf69b3016b1d44f59eabb193013b685b, user_id=98e93e7684414365b10306267721c229, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.631995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c1d8d-9b3d-4f73-9105-8854631925e1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.645659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 946/1891] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:11 2022] PUT /identity/v3/projects/bf69b3016b1d44f59eabb193013b685b/users/98e93e7684414365b10306267721c229/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:11.651556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6086f7a800614e4ba1c1139c604cab0c', 'request_id': 'req-86de16b9-79ac-42d0-ac18-2742c03e23cf'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:11.661419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6086f7a800614e4ba1c1139c604cab0c', 'request_id': 'req-86de16b9-79ac-42d0-ac18-2742c03e23cf'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:11.665973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.666655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.667053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.674532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6086f7a800614e4ba1c1139c604cab0c', 'request_id': 'req-86de16b9-79ac-42d0-ac18-2742c03e23cf'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:11.692034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] The token cache is being invalidate because project 6086f7a800614e4ba1c1139c604cab0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:11.692985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:11.693765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-86de16b9-79ac-42d0-ac18-2742c03e23cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:11.695938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 946/1892] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:11 2022] DELETE /identity/v3/projects/6086f7a800614e4ba1c1139c604cab0c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:11.701677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ea01-d859-4fb3-a48e-ffffde887ba8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.702080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ea01-d859-4fb3-a48e-ffffde887ba8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.702388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ea01-d859-4fb3-a48e-ffffde887ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.726560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5a76ea01-d859-4fb3-a48e-ffffde887ba8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:11.728545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5a76ea01-d859-4fb3-a48e-ffffde887ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:11.745651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] MFA Rules not processed for user `98e93e7684414365b10306267721c229`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:11.746919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:11.770165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:11.832005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 947/1893] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:11.842639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d86ac8-e838-428b-b8b3-8c26c745c876 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.842970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d86ac8-e838-428b-b8b3-8c26c745c876 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.843604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d86ac8-e838-428b-b8b3-8c26c745c876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.847438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 947/1894] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:11.859496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.861039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:11.873163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c5d86ac8-e838-428b-b8b3-8c26c745c876 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:11.875578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c5d86ac8-e838-428b-b8b3-8c26c745c876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:11.899462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 948/1895] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:11.904938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d86ac8-e838-428b-b8b3-8c26c745c876 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.923119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d86ac8-e838-428b-b8b3-8c26c745c876 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.929607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.930440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.930588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.931082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.931754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:11.932073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.933835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:11.949295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 949/1896] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:58:11.953849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:11.966096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362bd8-bca4-4a3a-82c1-bb2b75fc73ac admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:11.973473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:11.974388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:11.975327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:11.975519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:11.976330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:11.981450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.005613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753a4f8-ba67-4439-987a-19f9afca4082 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.017146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 950/1897] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:11 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:58:12.019422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.019730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.020102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.020389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.020720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.033365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.035393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.048867 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 948/1898] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:12.119866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.123933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:12.144683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913138-8436-4b94-851e-1dc8a85b53b0 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.150440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.151348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.151640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.152176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] PATH_INFO: `/v3/users/9dd36f8bebb64863b6804b88cbd2f785` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.153558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd36f8bebb64863b6804b88cbd2f785)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.155470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.155773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24320b-4cfd-4483-9667-c8c06e45bc0e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.157256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:12.163282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.164167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.164463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.164746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.165697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.169684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.210394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 951/1899] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:12 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:12.221868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.252012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47226507-0360-499f-8016-ad174d8e9bbf admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.263563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.264663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.264960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.265375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.266322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.268153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.288643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 952/1900] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:58:12.289812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.307507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4c61e-e11f-423f-8b20-ec5c60f4d2ce admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.317007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.317733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.318023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.318308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.319256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.321904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.337630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 953/1901] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:12.344616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd36f8bebb64863b6804b88cbd2f785'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:12.345892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd36f8bebb64863b6804b88cbd2f785'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:12.348307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.353781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:12.359642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd36f8bebb64863b6804b88cbd2f785'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:12.363098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd36f8bebb64863b6804b88cbd2f785'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:12.376620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-49c5d155-a942-4cf7-ad05-7a63ca6a6ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd36f8bebb64863b6804b88cbd2f785'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:12.396327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665d1dd-3ecf-4018-81b8-a5c0b9041107 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.409445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b5edc-b696-46c0-834b-edb559626f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.410364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b5edc-b696-46c0-834b-edb559626f5e admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.411144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b5edc-b696-46c0-834b-edb559626f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.411442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b5edc-b696-46c0-834b-edb559626f5e admin admin] PATH_INFO: `/v3/projects/d9bd1eb34fd44dcb879f1145312a9760/users/6ddbfb12a39f42c49f6f8c33459cf5ca/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.434180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 949/1902] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:12 2022] DELETE /identity/v3/users/9dd36f8bebb64863b6804b88cbd2f785 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:12.439639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.439907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.440179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.452090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b5edc-b696-46c0-834b-edb559626f5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9bd1eb34fd44dcb879f1145312a9760, user_id=6ddbfb12a39f42c49f6f8c33459cf5ca, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.460197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b5edc-b696-46c0-834b-edb559626f5e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.470398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:12.471820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:12.488427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 954/1903] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:12 2022] PUT /identity/v3/projects/d9bd1eb34fd44dcb879f1145312a9760/users/6ddbfb12a39f42c49f6f8c33459cf5ca/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:12.500318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.500621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.500987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.544893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] MFA Rules not processed for user `6ddbfb12a39f42c49f6f8c33459cf5ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:12.546206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:12.569483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:12.576112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 950/1904] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:12.587735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.589279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:12.636403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 955/1905] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:12.647891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.649401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:12.657199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cee2b-1865-4604-b78e-3a8c6931f227 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.661348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.662020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.662343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.662577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.673137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.674750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.686199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 951/1906] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:12.734469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c79f3-2f69-49c9-8f9f-1a7e89132ed7 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:12.738687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:12.739435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:12.739807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:12.740125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:12.750316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:12.751879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:12.762460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 956/1907] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:12.997969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:12.999993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:13.020968 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9427c-5a9e-4fc8-8229-46387eb327d7 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:13.029699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:13.031115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:13.031442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:13.031753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] PATH_INFO: `/v3/projects/758d689409144c7b9e2a2fc6600cc297` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:13.033236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] RBAC: Authorizing `identity:delete_project(project_id=758d689409144c7b9e2a2fc6600cc297)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:13.035722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:13.065646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:13.107030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758d689409144c7b9e2a2fc6600cc297', 'request_id': 'req-fd9600d9-ba03-4182-829e-31b0447c814c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.114153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758d689409144c7b9e2a2fc6600cc297', 'request_id': 'req-fd9600d9-ba03-4182-829e-31b0447c814c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.123208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758d689409144c7b9e2a2fc6600cc297', 'request_id': 'req-fd9600d9-ba03-4182-829e-31b0447c814c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.149565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] The token cache is being invalidate because project 758d689409144c7b9e2a2fc6600cc297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:13.150371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.151030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.152951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 952/1908] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:12 2022] DELETE /identity/v3/projects/758d689409144c7b9e2a2fc6600cc297 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:13.157502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:13.168594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04288604-1380-41c6-9814-422d2c4ab7fa service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:13.316678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:13.317936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:13.318162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:13.318672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] PATH_INFO: `/v3/users/98e93e7684414365b10306267721c229` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:13.320035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e93e7684414365b10306267721c229)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:13.322850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:13.350803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:13.386001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e93e7684414365b10306267721c229'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.387070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e93e7684414365b10306267721c229'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.387929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e93e7684414365b10306267721c229'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.403197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e93e7684414365b10306267721c229'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.423144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-32d3dbba-ddde-4931-9b46-5942baa29d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e93e7684414365b10306267721c229'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:13.468372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 957/1909] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:13 2022] DELETE /identity/v3/users/98e93e7684414365b10306267721c229 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:14.017151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:14.019540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:14.141918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9600d9-ba03-4182-829e-31b0447c814c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:14.245390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:14.246300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:14.246586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:14.246866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] PATH_INFO: `/v3/projects/bf69b3016b1d44f59eabb193013b685b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:14.248249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf69b3016b1d44f59eabb193013b685b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:14.250225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:14.362011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf69b3016b1d44f59eabb193013b685b', 'request_id': 'req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:14.368875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:14.379025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf69b3016b1d44f59eabb193013b685b', 'request_id': 'req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:14.406082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf69b3016b1d44f59eabb193013b685b', 'request_id': 'req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:14.447500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] The token cache is being invalidate because project bf69b3016b1d44f59eabb193013b685b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:14.447883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:14.448577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8793f9f5-1ddd-47db-bebd-51f2cb2aa557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:14.453866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 953/1910] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:14 2022] DELETE /identity/v3/projects/bf69b3016b1d44f59eabb193013b685b => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:14.455964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e0a17-fdf2-4c0c-83a4-906639690f7b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:14.456904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e0a17-fdf2-4c0c-83a4-906639690f7b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:14.457355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e0a17-fdf2-4c0c-83a4-906639690f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:14.561110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-434e0a17-fdf2-4c0c-83a4-906639690f7b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:14.562682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-434e0a17-fdf2-4c0c-83a4-906639690f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:14.659679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 958/1911] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:14.668010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39339-563e-4d6a-a038-8b6fbef1c55b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:14.668352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39339-563e-4d6a-a038-8b6fbef1c55b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:14.668581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39339-563e-4d6a-a038-8b6fbef1c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:14.766653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-79e39339-563e-4d6a-a038-8b6fbef1c55b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:14.772307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-79e39339-563e-4d6a-a038-8b6fbef1c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:14.797197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 954/1912] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:14.801829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e0a17-fdf2-4c0c-83a4-906639690f7b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:14.812946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e0a17-fdf2-4c0c-83a4-906639690f7b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:14.818325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:14.819013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:14.819330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:14.819592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:14.820319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:14.820636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:14.822490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:14.836101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 959/1913] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:58:14.842073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e39339-563e-4d6a-a038-8b6fbef1c55b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:14.851827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e39339-563e-4d6a-a038-8b6fbef1c55b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:14.859403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:14.860170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:14.860456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:14.860876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:14.861487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:14.863568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:14.934696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 955/1914] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:14 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:14.935959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:14.948488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545547f5-68c8-4eaa-93b9-81e6e117f954 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.035340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.036437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.036746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.037029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.037723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.039722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.053601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:15.073873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 960/1915] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:14 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:15.078046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.094004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ef98f-c31f-4eca-b8ba-bf7ce238c9d2 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.100767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.101945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.102409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.102873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.103790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.106081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.122640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 956/1916] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:15.127164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.138874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6a89c-a5bf-4110-9f5a-ab17e83fff26 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.149393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47858250-74e1-4601-b9a1-7c97dde33eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.150844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858250-74e1-4601-b9a1-7c97dde33eaf admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.151513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858250-74e1-4601-b9a1-7c97dde33eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.152541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858250-74e1-4601-b9a1-7c97dde33eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.154217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47858250-74e1-4601-b9a1-7c97dde33eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.158007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47858250-74e1-4601-b9a1-7c97dde33eaf admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.173256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 961/1917] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:58:15.177586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.189487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66273e-b4f4-447c-bee2-d2c3fa5f5663 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.197332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.198774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.199231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.199679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] PATH_INFO: `/v3/projects/778e03c28bd740c09f617990431a3166/users/523faac57f5043e49c806d595336e5bb/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.223087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=778e03c28bd740c09f617990431a3166, user_id=523faac57f5043e49c806d595336e5bb, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.225861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.243259 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 957/1918] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:15 2022] PUT /identity/v3/projects/778e03c28bd740c09f617990431a3166/users/523faac57f5043e49c806d595336e5bb/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:15.254499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50f44-a158-42ab-be9b-ef81d670e697 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.254795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50f44-a158-42ab-be9b-ef81d670e697 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.255081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50f44-a158-42ab-be9b-ef81d670e697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.288726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fea50f44-a158-42ab-be9b-ef81d670e697 None None] MFA Rules not processed for user `523faac57f5043e49c806d595336e5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:15.290735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fea50f44-a158-42ab-be9b-ef81d670e697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:15.369160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 962/1919] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:15.373794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.444213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a140be68-79df-4677-add9-dd57fae76e2c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.449155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.449831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.450116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.450423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.451122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.453052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.483576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 958/1920] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:15 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:15.487687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50f44-a158-42ab-be9b-ef81d670e697 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.499801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50f44-a158-42ab-be9b-ef81d670e697 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.560477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.561204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.561485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.561763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.562607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.564585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.589851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 963/1921] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:15 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:15.594490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.611806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b1b7e-02b1-4a07-bdad-471727c69f05 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.616481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.617191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.617456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.617727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.618454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.620041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.631478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 959/1922] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:15.636043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.648158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3131c65-4008-4fa5-a849-020808c63cd8 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.654906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.655917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.656206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.656487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] PATH_INFO: `/v3/projects/145acd98c5054239bebfffa4d73acafb/users/05bf6fc6c5bb48348929c3de921c7f51/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.673360 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145acd98c5054239bebfffa4d73acafb, user_id=05bf6fc6c5bb48348929c3de921c7f51, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.676053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.695424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 964/1923] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:15 2022] PUT /identity/v3/projects/145acd98c5054239bebfffa4d73acafb/users/05bf6fc6c5bb48348929c3de921c7f51/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:15.700142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.717313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de71b8-ac24-4a5d-bbfe-d32b5d22039e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.769545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:15.785350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf2a73-849c-4d6c-9e02-94efb1ae9b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.787605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2a73-849c-4d6c-9e02-94efb1ae9b6d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.787909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2a73-849c-4d6c-9e02-94efb1ae9b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.788194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf2a73-849c-4d6c-9e02-94efb1ae9b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.788969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf2a73-849c-4d6c-9e02-94efb1ae9b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.791204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf2a73-849c-4d6c-9e02-94efb1ae9b6d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.810859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 960/1924] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:15.816245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:15.837753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7cbee-5fd7-48df-8d63-1c6944ae5a55 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:15.843093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:15.844201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.844488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.844760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] PATH_INFO: `/v3/projects/145acd98c5054239bebfffa4d73acafb/users/05bf6fc6c5bb48348929c3de921c7f51/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.846899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145acd98c5054239bebfffa4d73acafb, user_id=05bf6fc6c5bb48348929c3de921c7f51, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:15.857581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:15.868933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 965/1925] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:15 2022] PUT /identity/v3/projects/145acd98c5054239bebfffa4d73acafb/users/05bf6fc6c5bb48348929c3de921c7f51/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:15.880283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bbe6c-d6ee-4ae7-87ed-a28e2967d14c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:15.880576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bbe6c-d6ee-4ae7-87ed-a28e2967d14c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:15.880886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bbe6c-d6ee-4ae7-87ed-a28e2967d14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:15.915403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a49bbe6c-d6ee-4ae7-87ed-a28e2967d14c None None] MFA Rules not processed for user `05bf6fc6c5bb48348929c3de921c7f51`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:15.917687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a49bbe6c-d6ee-4ae7-87ed-a28e2967d14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:15.998603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 961/1926] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:16.010973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:16.018248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:16.052901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:16.105733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2a07-2fbf-4d86-855c-03e3225e0292 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:16.109421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:16.110100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:16.110361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:16.110639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:16.122108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:16.123753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:16.139167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 966/1927] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:16.247232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bbe6c-d6ee-4ae7-87ed-a28e2967d14c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:16.253638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bbe6c-d6ee-4ae7-87ed-a28e2967d14c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:16.261478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2701f1-91eb-4e1b-abe9-82e7326d5933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:16.262354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2701f1-91eb-4e1b-abe9-82e7326d5933 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:16.262695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2701f1-91eb-4e1b-abe9-82e7326d5933 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:16.262971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2701f1-91eb-4e1b-abe9-82e7326d5933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:16.357923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2701f1-91eb-4e1b-abe9-82e7326d5933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:16.359473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2701f1-91eb-4e1b-abe9-82e7326d5933 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:16.370345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 962/1928] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:26.130433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=21 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:26.364760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=67 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:27.568619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:27.570140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:27.643107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636f12a-cf53-4427-af3c-058e0c6dade6 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:27.648481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:27.649489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:27.649780 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:27.650066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] PATH_INFO: `/v3/users/eb5996b5f1bf45778e17fa5ea4283303` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:27.651581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5996b5f1bf45778e17fa5ea4283303)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:27.653920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:27.714807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5996b5f1bf45778e17fa5ea4283303'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:27.715465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5996b5f1bf45778e17fa5ea4283303'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:27.717158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5996b5f1bf45778e17fa5ea4283303'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:27.725865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5996b5f1bf45778e17fa5ea4283303'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:27.739375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5996b5f1bf45778e17fa5ea4283303'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:27.761491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 967/1929] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:27 2022] DELETE /identity/v3/users/eb5996b5f1bf45778e17fa5ea4283303 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:27.766621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:27.767330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:27.767330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:27.786959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:27.788307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:27.894422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 963/1930] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:27.906410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:27.908259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:28.155157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b9823-cc01-4aa7-8f1b-e2e461fa2fec admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:28.159220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:28.159900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:28.160184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:28.160462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:28.180191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:28.182768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:28.201480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 968/1931] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:28.701562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:28.703128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:28.728794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be93c2-f4a6-4446-9afd-7609f49d3fd9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:28.743523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:28.744329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:28.744608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:28.745336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] PATH_INFO: `/v3/projects/2c226754c0de49c491c2d16963863689` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:28.746971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c226754c0de49c491c2d16963863689)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:28.749471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:28.832230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c226754c0de49c491c2d16963863689', 'request_id': 'req-c24065b3-2825-46a2-95f5-edb58b59823d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:28.839261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c226754c0de49c491c2d16963863689', 'request_id': 'req-c24065b3-2825-46a2-95f5-edb58b59823d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:28.846864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c226754c0de49c491c2d16963863689', 'request_id': 'req-c24065b3-2825-46a2-95f5-edb58b59823d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:28.864098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] The token cache is being invalidate because project 2c226754c0de49c491c2d16963863689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:28.864554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:28.865621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:28.868188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 964/1932] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:28 2022] DELETE /identity/v3/projects/2c226754c0de49c491c2d16963863689 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:28.872512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:28.878065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:28.885597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eaaca5-da85-4abd-9cc6-7cb701c0aafb service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:28.960406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:28.962428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:28.962849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:28.963351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] PATH_INFO: `/v3/users/8021d5cccbaf4f06a8d0e7275bd7ec89` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:28.965559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8021d5cccbaf4f06a8d0e7275bd7ec89)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:28.969184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:29.036762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8021d5cccbaf4f06a8d0e7275bd7ec89'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.037124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8021d5cccbaf4f06a8d0e7275bd7ec89'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.037917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8021d5cccbaf4f06a8d0e7275bd7ec89'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.046810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8021d5cccbaf4f06a8d0e7275bd7ec89'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.064425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9c5edb21-8689-4cca-9715-a7c7f8c8c41a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8021d5cccbaf4f06a8d0e7275bd7ec89'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.083703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 969/1933] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:28 2022] DELETE /identity/v3/users/8021d5cccbaf4f06a8d0e7275bd7ec89 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:29.415746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:29.417552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:29.437071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24065b3-2825-46a2-95f5-edb58b59823d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:29.517804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:29.518593 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:29.518872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:29.519145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] PATH_INFO: `/v3/projects/ae9442091e12479295a3d692a538749b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:29.520649 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae9442091e12479295a3d692a538749b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:29.522646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:29.557418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:29.583726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae9442091e12479295a3d692a538749b', 'request_id': 'req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.598171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae9442091e12479295a3d692a538749b', 'request_id': 'req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.607418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae9442091e12479295a3d692a538749b', 'request_id': 'req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.625002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] The token cache is being invalidate because project ae9442091e12479295a3d692a538749b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:29.625561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.626538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-78f8b500-fa57-4bce-9dcb-aeb63b529aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:29.629291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 965/1934] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:29 2022] DELETE /identity/v3/projects/ae9442091e12479295a3d692a538749b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:29.635145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c7cae-034d-40ca-8bd5-12c054c57b8d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:29.635500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c7cae-034d-40ca-8bd5-12c054c57b8d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:29.635777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c7cae-034d-40ca-8bd5-12c054c57b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:29.659931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-d27c7cae-034d-40ca-8bd5-12c054c57b8d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:29.661223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d27c7cae-034d-40ca-8bd5-12c054c57b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:29.737767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 970/1935] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:29.746288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4110d-2451-4356-96fb-34a223d02b1e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:29.746614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4110d-2451-4356-96fb-34a223d02b1e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:29.746896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4110d-2451-4356-96fb-34a223d02b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:29.770945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-53a4110d-2451-4356-96fb-34a223d02b1e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:29.772710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-53a4110d-2451-4356-96fb-34a223d02b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:29.786816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 966/1936] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:29.795669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c7cae-034d-40ca-8bd5-12c054c57b8d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:29.811611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c7cae-034d-40ca-8bd5-12c054c57b8d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:29.818159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:29.819371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:29.820265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:29.820831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:29.822754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:29.823426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:29.826590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:29.848417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 971/1937] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:58:29.853296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4110d-2451-4356-96fb-34a223d02b1e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:29.864124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4110d-2451-4356-96fb-34a223d02b1e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:29.870531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:29.871328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:29.871714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:29.872060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:29.873281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:29.875257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:29.915453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 967/1938] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:29 2022] POST /identity/v3/projects => generated 357 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:29.916738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:29.928312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44e27e-c7f0-4c91-9bb9-ad1f9be28454 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:29.933876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:30.008211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.008211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.008211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.008211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.009637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.011653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.037333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 972/1939] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:29 2022] POST /identity/v3/users => generated 279 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:30.041519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.052323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104932c-d3b1-4262-a77a-46fadfbc10cf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.059310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.060261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.060737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.061213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.061969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.063596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.075432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 968/1940] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:30.079876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.092970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aadc4-ea26-431c-b2b3-0a05ee0fd2eb admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.099882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cf64c-34ee-482e-905a-41facaa1993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.100580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf64c-34ee-482e-905a-41facaa1993c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.100896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf64c-34ee-482e-905a-41facaa1993c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.101241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf64c-34ee-482e-905a-41facaa1993c admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.102491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cf64c-34ee-482e-905a-41facaa1993c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.103638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cf64c-34ee-482e-905a-41facaa1993c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.121211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 973/1941] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:58:30.125662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.140414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c95cb-d77b-4be0-872a-1ecb817f9a2a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.147363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.148286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.148569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.148868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] PATH_INFO: `/v3/projects/9289e2bdb0334c6a9557bd4acbc09d04/users/bdf8cbaa92e14926b1633fa793e0cd1e/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.171210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9289e2bdb0334c6a9557bd4acbc09d04, user_id=bdf8cbaa92e14926b1633fa793e0cd1e, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.173960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.192413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 969/1942] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:30 2022] PUT /identity/v3/projects/9289e2bdb0334c6a9557bd4acbc09d04/users/bdf8cbaa92e14926b1633fa793e0cd1e/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:30.210369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827080e-6874-4ee2-ab20-807eb6a38cad None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.210670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827080e-6874-4ee2-ab20-807eb6a38cad None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.210983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827080e-6874-4ee2-ab20-807eb6a38cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.245105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5827080e-6874-4ee2-ab20-807eb6a38cad None None] MFA Rules not processed for user `bdf8cbaa92e14926b1633fa793e0cd1e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:30.246927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5827080e-6874-4ee2-ab20-807eb6a38cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:30.335723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 974/1943] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:30.341631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.407243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82eb608-f8ba-4c74-947f-14a3317b7227 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.412143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.414092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.414092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.414092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.414092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.416550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.444706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 970/1944] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:30 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:58:30.448259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827080e-6874-4ee2-ab20-807eb6a38cad None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.459406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827080e-6874-4ee2-ab20-807eb6a38cad None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.525817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.526510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.526794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.527366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.528027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.530060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.562250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 975/1945] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:30 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:30.563645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.577480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa0bbf-31f5-4a2a-bb09-3da379028e45 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.584346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.585053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.585343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.585623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.587655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.589420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.604498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 971/1946] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:30.607022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.631010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e3a9e-7e11-4f18-af29-d709bddd4750 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.637585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.638676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.638958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.639234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] PATH_INFO: `/v3/projects/86bc9e0bdcaa4efebde3e2c4fa47615d/users/eb3a251363a8455ea327f77b86ce482e/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.641896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:30.659355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bc9e0bdcaa4efebde3e2c4fa47615d, user_id=eb3a251363a8455ea327f77b86ce482e, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.665420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.685044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 976/1947] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:30 2022] PUT /identity/v3/projects/86bc9e0bdcaa4efebde3e2c4fa47615d/users/eb3a251363a8455ea327f77b86ce482e/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:30.689408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.705654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e600d232-f2a6-4e88-94a9-4bd16adef3b5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.727752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:30.769039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b751df5-605e-454f-b0a9-59dff6aaa33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.769717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b751df5-605e-454f-b0a9-59dff6aaa33f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.770002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b751df5-605e-454f-b0a9-59dff6aaa33f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.770280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b751df5-605e-454f-b0a9-59dff6aaa33f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.771017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b751df5-605e-454f-b0a9-59dff6aaa33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.772627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b751df5-605e-454f-b0a9-59dff6aaa33f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.787168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 972/1948] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:30.788900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.799748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33df75-42a3-44b5-8cb9-91a38f0464db admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:30.816576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:30.817380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.817672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.817957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] PATH_INFO: `/v3/projects/86bc9e0bdcaa4efebde3e2c4fa47615d/users/eb3a251363a8455ea327f77b86ce482e/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.820066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bc9e0bdcaa4efebde3e2c4fa47615d, user_id=eb3a251363a8455ea327f77b86ce482e, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:30.823371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:30.835415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 977/1949] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:30 2022] PUT /identity/v3/projects/86bc9e0bdcaa4efebde3e2c4fa47615d/users/eb3a251363a8455ea327f77b86ce482e/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:30.846062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:30.846249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:30.846540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:30.875577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] MFA Rules not processed for user `eb3a251363a8455ea327f77b86ce482e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:30.876815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:30.974917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 973/1950] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:30.985337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:30.987381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:31.080994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df69ee-c375-4ade-b412-06db01a2f64d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:31.090551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:31.091241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:31.091525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:31.091913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:31.102264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:31.104128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:31.114720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 978/1951] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:31.196423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:31.198641 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:31.219324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efee7ae-846c-4b0b-a252-b5a85f1595ca None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:31.225292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb5b74-4629-49fe-b676-108a4b24a94a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:31.226404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb5b74-4629-49fe-b676-108a4b24a94a service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:31.226779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb5b74-4629-49fe-b676-108a4b24a94a service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:31.227436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb5b74-4629-49fe-b676-108a4b24a94a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:31.299920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb5b74-4629-49fe-b676-108a4b24a94a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:31.301425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb5b74-4629-49fe-b676-108a4b24a94a service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:31.311675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 974/1952] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:33.515463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:33.517358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:33.601468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c846dd-6738-466d-a4f9-57b0a05fe5a2 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:33.608117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:33.609228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:33.610579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:33.610889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] PATH_INFO: `/v3/users/523faac57f5043e49c806d595336e5bb` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:33.612403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=523faac57f5043e49c806d595336e5bb)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:33.628398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:33.711226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523faac57f5043e49c806d595336e5bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:33.711529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523faac57f5043e49c806d595336e5bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:33.712492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523faac57f5043e49c806d595336e5bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:33.730152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523faac57f5043e49c806d595336e5bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:33.747152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523faac57f5043e49c806d595336e5bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:33.750105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:33.783397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 979/1953] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:33 2022] DELETE /identity/v3/users/523faac57f5043e49c806d595336e5bb => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:33.788804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:33.789341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:33.789417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:33.807498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:33.809014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:33.860273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:33.888215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 975/1954] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:33.899897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:33.901783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:33.978470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8292a05-7ada-4052-a899-3e1f9ec3b54c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:33.982683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:33.984017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:33.984421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:33.984743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:34.006639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:34.008118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:34.035486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 980/1955] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:34.669770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:34.671356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:34.690727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22abbe8-cb8b-454f-8f1a-1c8f0a604382 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:34.698613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:34.699717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:34.700171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:34.700586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] PATH_INFO: `/v3/projects/778e03c28bd740c09f617990431a3166` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:34.702516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778e03c28bd740c09f617990431a3166)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:34.704990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:34.792609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778e03c28bd740c09f617990431a3166', 'request_id': 'req-7a0d735c-4b46-4a55-912c-03017303c9b8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:34.802538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778e03c28bd740c09f617990431a3166', 'request_id': 'req-7a0d735c-4b46-4a55-912c-03017303c9b8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:34.822739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778e03c28bd740c09f617990431a3166', 'request_id': 'req-7a0d735c-4b46-4a55-912c-03017303c9b8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:34.838146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] The token cache is being invalidate because project 778e03c28bd740c09f617990431a3166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:34.838691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:34.840085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:34.841724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 976/1956] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:34 2022] DELETE /identity/v3/projects/778e03c28bd740c09f617990431a3166 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:34.848795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:34.860105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec78c82-5775-4276-82a3-df18a4a60c43 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:34.936965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:34.937944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:34.938234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:34.938522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] PATH_INFO: `/v3/users/05bf6fc6c5bb48348929c3de921c7f51` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:34.939900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] RBAC: Authorizing `identity:delete_user(user_id=05bf6fc6c5bb48348929c3de921c7f51)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:34.942058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:34.998740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05bf6fc6c5bb48348929c3de921c7f51'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:34.999422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05bf6fc6c5bb48348929c3de921c7f51'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.000040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05bf6fc6c5bb48348929c3de921c7f51'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.009610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05bf6fc6c5bb48348929c3de921c7f51'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.022066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5615f3d7-f821-4230-860d-8f8e3353698e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05bf6fc6c5bb48348929c3de921c7f51'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.036802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 981/1957] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:34 2022] DELETE /identity/v3/users/05bf6fc6c5bb48348929c3de921c7f51 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:35.352793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:35.354451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:35.370781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d735c-4b46-4a55-912c-03017303c9b8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:35.449448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:35.451649 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:35.452796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:35.453296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] PATH_INFO: `/v3/projects/145acd98c5054239bebfffa4d73acafb` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:35.455393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145acd98c5054239bebfffa4d73acafb)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:35.458549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:35.527021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145acd98c5054239bebfffa4d73acafb', 'request_id': 'req-6700bfc3-1ebb-49ed-a818-7fa8eea64638'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.539327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145acd98c5054239bebfffa4d73acafb', 'request_id': 'req-6700bfc3-1ebb-49ed-a818-7fa8eea64638'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.557672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145acd98c5054239bebfffa4d73acafb', 'request_id': 'req-6700bfc3-1ebb-49ed-a818-7fa8eea64638'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.570477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:35.577590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] The token cache is being invalidate because project 145acd98c5054239bebfffa4d73acafb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:35.577995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.579638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6700bfc3-1ebb-49ed-a818-7fa8eea64638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:35.590897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cffe56-6e68-4e77-8e4e-0cad163f8228 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:35.591450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cffe56-6e68-4e77-8e4e-0cad163f8228 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:35.591865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cffe56-6e68-4e77-8e4e-0cad163f8228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:35.593992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 977/1958] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:35 2022] DELETE /identity/v3/projects/145acd98c5054239bebfffa4d73acafb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:35.623297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-59cffe56-6e68-4e77-8e4e-0cad163f8228 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:35.626073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-59cffe56-6e68-4e77-8e4e-0cad163f8228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:35.649999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:35.730438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 982/1959] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:35.738625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d72d0-143a-4cc3-aaa7-ebacdba52173 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:35.739071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d72d0-143a-4cc3-aaa7-ebacdba52173 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:35.739363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d72d0-143a-4cc3-aaa7-ebacdba52173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:35.773635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-eb6d72d0-143a-4cc3-aaa7-ebacdba52173 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:35.776137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-eb6d72d0-143a-4cc3-aaa7-ebacdba52173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:35.798413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 978/1960] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:35.799256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cffe56-6e68-4e77-8e4e-0cad163f8228 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:35.815524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cffe56-6e68-4e77-8e4e-0cad163f8228 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:35.822686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:35.823019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:35.823319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:35.823613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:35.824364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:35.824683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:35.826640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:35.859064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 983/1961] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:58:35.866237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d72d0-143a-4cc3-aaa7-ebacdba52173 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:35.890427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d72d0-143a-4cc3-aaa7-ebacdba52173 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:35.898574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:35.899996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:35.900421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:35.900766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:35.901637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:35.904211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:35.974947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 979/1962] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:35 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:35.980606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:35.996038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487700a-ab7b-4bba-a4be-9969604fed80 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:36.104912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:36.104912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:36.104912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:36.104912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:36.105571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:36.105571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:36.153069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 984/1963] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:35 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:58:36.156807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:36.176377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81982b-3403-4511-aac0-e2a8fdadce63 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:36.184894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:36.186123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:36.186720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:36.187210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:36.188586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:36.191507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:36.210727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 980/1964] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:36.215773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:36.229323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbddf84-67e0-4639-9ac7-4dee614c59e9 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:36.234727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51b700-e5b3-4886-aac7-27bb6e41f7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:36.235417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51b700-e5b3-4886-aac7-27bb6e41f7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:36.235705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51b700-e5b3-4886-aac7-27bb6e41f7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:36.235995 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51b700-e5b3-4886-aac7-27bb6e41f7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:36.236739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51b700-e5b3-4886-aac7-27bb6e41f7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:36.238425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51b700-e5b3-4886-aac7-27bb6e41f7a3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:36.250921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 985/1965] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:36.255346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:36.272578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60e475-2a1c-4f98-ace2-e1f336b6a00e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:36.279683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:36.280646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:36.280862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:36.281153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] PATH_INFO: `/v3/projects/7206ab05cd174bae9d62eebef69af57f/users/3e207b9592764083975305553b948172/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:36.298447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7206ab05cd174bae9d62eebef69af57f, user_id=3e207b9592764083975305553b948172, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:36.303950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:36.322720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 981/1966] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:36 2022] PUT /identity/v3/projects/7206ab05cd174bae9d62eebef69af57f/users/3e207b9592764083975305553b948172/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:36.334568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:36.334798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:36.335072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:36.375638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] MFA Rules not processed for user `3e207b9592764083975305553b948172`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:36.376894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:36.447952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 986/1967] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:36.461928 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:36.463740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:36.540960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30f7ae-3923-43a4-b8c2-7841184ed428 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:36.544677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca470d1-c9d7-47ba-a0f0-32695aed3af7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:36.545399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca470d1-c9d7-47ba-a0f0-32695aed3af7 service glance] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:36.545679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca470d1-c9d7-47ba-a0f0-32695aed3af7 service glance] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:36.545960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca470d1-c9d7-47ba-a0f0-32695aed3af7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:36.555293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca470d1-c9d7-47ba-a0f0-32695aed3af7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:36.556732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca470d1-c9d7-47ba-a0f0-32695aed3af7 service glance] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:36.568006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 982/1968] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:58:46.446005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:46.561897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:46.629476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:46.631816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:46.724295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1112dbd-59ad-48a7-8138-d33e51398d41 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:46.729180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:46.730070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:46.730393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:46.730679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] PATH_INFO: `/v3/users/6ddbfb12a39f42c49f6f8c33459cf5ca` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:46.731915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddbfb12a39f42c49f6f8c33459cf5ca)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:46.733916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:46.809829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddbfb12a39f42c49f6f8c33459cf5ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:46.810505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddbfb12a39f42c49f6f8c33459cf5ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:46.811252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddbfb12a39f42c49f6f8c33459cf5ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:46.819329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddbfb12a39f42c49f6f8c33459cf5ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:46.834680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddbfb12a39f42c49f6f8c33459cf5ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:46.853687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 987/1969] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:46 2022] DELETE /identity/v3/users/6ddbfb12a39f42c49f6f8c33459cf5ca => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:46.858566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:46.858881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:46.859231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:46.877438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:46.878848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:46.950401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 983/1970] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:46.960468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:46.962557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:47.030284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607454c-08d3-4522-9863-9ba281f7e6ce admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:47.034040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db567a07-de6f-40e8-86c2-d00d64c2aa66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:47.034704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567a07-de6f-40e8-86c2-d00d64c2aa66 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:47.035086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567a07-de6f-40e8-86c2-d00d64c2aa66 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:47.035267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db567a07-de6f-40e8-86c2-d00d64c2aa66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:47.045900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567a07-de6f-40e8-86c2-d00d64c2aa66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:47.047475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db567a07-de6f-40e8-86c2-d00d64c2aa66 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:47.058669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 988/1971] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:47.441126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:47.442519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:47.471794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e835e62-f94e-4a0c-b7c0-6428566f0731 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:47.481647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:47.481647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:47.481647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:47.481647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] PATH_INFO: `/v3/projects/d9bd1eb34fd44dcb879f1145312a9760` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:47.482276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9bd1eb34fd44dcb879f1145312a9760)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:47.483622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:47.540255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9bd1eb34fd44dcb879f1145312a9760', 'request_id': 'req-fa7e61ee-6121-4c76-abe4-f5c553ad96af'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:47.546581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9bd1eb34fd44dcb879f1145312a9760', 'request_id': 'req-fa7e61ee-6121-4c76-abe4-f5c553ad96af'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:47.555997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9bd1eb34fd44dcb879f1145312a9760', 'request_id': 'req-fa7e61ee-6121-4c76-abe4-f5c553ad96af'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:47.571510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] The token cache is being invalidate because project d9bd1eb34fd44dcb879f1145312a9760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:47.572135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:47.572662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fa7e61ee-6121-4c76-abe4-f5c553ad96af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:47.574849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 984/1972] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:47 2022] DELETE /identity/v3/projects/d9bd1eb34fd44dcb879f1145312a9760 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:47.583320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59cbc-78d3-4663-83b9-fdad6a899692 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:47.583896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59cbc-78d3-4663-83b9-fdad6a899692 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:47.583994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59cbc-78d3-4663-83b9-fdad6a899692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:47.614594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-44a59cbc-78d3-4663-83b9-fdad6a899692 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:47.615838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-44a59cbc-78d3-4663-83b9-fdad6a899692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:47.626769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:47.698928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 989/1973] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:58:47.708446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89526-8418-4b94-89c8-fb8126b0b9f2 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:47.708734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89526-8418-4b94-89c8-fb8126b0b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:47.709087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89526-8418-4b94-89c8-fb8126b0b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:47.730364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ade89526-8418-4b94-89c8-fb8126b0b9f2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:47.731573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ade89526-8418-4b94-89c8-fb8126b0b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:47.739739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 985/1974] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:47.746641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59cbc-78d3-4663-83b9-fdad6a899692 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:47.758592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59cbc-78d3-4663-83b9-fdad6a899692 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:47.764319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:47.765102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:47.765386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:47.765664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:47.766403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:47.766714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:47.768322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:47.783652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 990/1975] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:58:47.790900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade89526-8418-4b94-89c8-fb8126b0b9f2 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:47.804264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade89526-8418-4b94-89c8-fb8126b0b9f2 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:47.815654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:47.816712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:47.817171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:47.817443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:47.818175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:47.820142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:47.854069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 986/1976] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:47 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:47.859511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:47.873030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30742c62-aba3-428d-b049-fba71a30a6b6 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:47.967549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:47.968248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:47.968573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:47.968802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:47.969543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:47.971816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:47.995693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 991/1977] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:47 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:58:48.000347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:48.012451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d2133-7d68-40a6-87e9-832b7cf7cfd9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:48.017477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:48.018054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:48.018343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:48.018621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:48.019357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:48.021129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:48.033373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 987/1978] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:48.038578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:48.050583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fc258-f6b5-4100-82d9-92aba0df3637 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:48.057929 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deb6b2-4954-4017-9f7f-63e412a94e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:48.061364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deb6b2-4954-4017-9f7f-63e412a94e55 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:48.061364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deb6b2-4954-4017-9f7f-63e412a94e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:48.061364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deb6b2-4954-4017-9f7f-63e412a94e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:48.061364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deb6b2-4954-4017-9f7f-63e412a94e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:48.061747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deb6b2-4954-4017-9f7f-63e412a94e55 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:48.073195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 992/1979] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:48.081005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:48.094754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d807a03-714d-4e22-b332-c027b9ed88ee admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:48.099569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:48.100473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:48.100761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:48.101073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] PATH_INFO: `/v3/projects/22a75da564c3434ca8a1cff32b9edd4d/users/0030bdbe0d0f466a97b37f4f5a83b02f/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:48.117508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a75da564c3434ca8a1cff32b9edd4d, user_id=0030bdbe0d0f466a97b37f4f5a83b02f, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:48.120204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:48.143985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 988/1980] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:48 2022] PUT /identity/v3/projects/22a75da564c3434ca8a1cff32b9edd4d/users/0030bdbe0d0f466a97b37f4f5a83b02f/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:48.159007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6827-94e7-44a3-89b9-7b94ca72719d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:48.159455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6827-94e7-44a3-89b9-7b94ca72719d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:48.159761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f6827-94e7-44a3-89b9-7b94ca72719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:48.194436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a88f6827-94e7-44a3-89b9-7b94ca72719d None None] MFA Rules not processed for user `0030bdbe0d0f466a97b37f4f5a83b02f`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:48.196219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a88f6827-94e7-44a3-89b9-7b94ca72719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:48.265547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 993/1981] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:48.279308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:48.281095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:48.303997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:48.348564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a3607-12ff-408c-8195-4351dbe80423 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:48.352876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:48.353546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:48.353833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:48.354110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:48.363478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:48.364948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:48.374435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 989/1982] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:50.907967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f8c24-7f6f-49ad-881d-6935318e4a98 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:50.908666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f8c24-7f6f-49ad-881d-6935318e4a98 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:50.909277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f8c24-7f6f-49ad-881d-6935318e4a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:50.940043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a82f8c24-7f6f-49ad-881d-6935318e4a98 None None] MFA Rules not processed for user `0030bdbe0d0f466a97b37f4f5a83b02f`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:50.941985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a82f8c24-7f6f-49ad-881d-6935318e4a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:50.951903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 994/1983] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:54.107387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:54.109517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:54.202438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd505ac-1a59-4489-9a83-85d5e16856ff service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:54.207662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:54.208593 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:54.208923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:54.209196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] PATH_INFO: `/v3/users/bdf8cbaa92e14926b1633fa793e0cd1e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:54.210516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf8cbaa92e14926b1633fa793e0cd1e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:54.212538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:54.267035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf8cbaa92e14926b1633fa793e0cd1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:54.267721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf8cbaa92e14926b1633fa793e0cd1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:54.275924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf8cbaa92e14926b1633fa793e0cd1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:54.276708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf8cbaa92e14926b1633fa793e0cd1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:54.285258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf8cbaa92e14926b1633fa793e0cd1e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:54.305990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 990/1984] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:54 2022] DELETE /identity/v3/users/bdf8cbaa92e14926b1633fa793e0cd1e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:54.311331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:54.311912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:54.311912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:54.318592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:54.333849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:54.334988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:54.412921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 995/1985] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:54.423312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:54.424768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:54.489203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:54.513469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f258-d260-4f56-9a18-aca74bf1d2fa admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:54.517855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:54.518524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:54.518809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:54.519084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:54.530482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:54.531972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:54.542990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 991/1986] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:55.007104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:55.009440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:55.024758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f8da9-cd81-4f36-834d-23210a156215 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:55.030577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:55.032487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:55.038666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:55.038973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] PATH_INFO: `/v3/projects/9289e2bdb0334c6a9557bd4acbc09d04` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:55.044166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9289e2bdb0334c6a9557bd4acbc09d04)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:55.046434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:55.105169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9289e2bdb0334c6a9557bd4acbc09d04', 'request_id': 'req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.114924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9289e2bdb0334c6a9557bd4acbc09d04', 'request_id': 'req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.123666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9289e2bdb0334c6a9557bd4acbc09d04', 'request_id': 'req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.138320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] The token cache is being invalidate because project 9289e2bdb0334c6a9557bd4acbc09d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:55.138610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.139279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.141308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 996/1987] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:55 2022] DELETE /identity/v3/projects/9289e2bdb0334c6a9557bd4acbc09d04 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:58:55.146004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:55.161793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9df4a-d772-463d-a476-99e97841b107 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:55.271122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:55.272036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:55.272313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:55.272588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] PATH_INFO: `/v3/users/eb3a251363a8455ea327f77b86ce482e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:55.274654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb3a251363a8455ea327f77b86ce482e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:55.276967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:55.336179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb3a251363a8455ea327f77b86ce482e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.336179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb3a251363a8455ea327f77b86ce482e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.345958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb3a251363a8455ea327f77b86ce482e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.346959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb3a251363a8455ea327f77b86ce482e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.354428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f9c34f41-adf3-441d-8eca-f67fafe5dc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb3a251363a8455ea327f77b86ce482e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.379693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 992/1988] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:58:55 2022] DELETE /identity/v3/users/eb3a251363a8455ea327f77b86ce482e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:55.696343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:55.698635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:55.719036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bc91a-535b-46b2-bc4a-e7e7e3e3ee93 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:55.790627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:55.791509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:55.791790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:55.792062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] PATH_INFO: `/v3/projects/86bc9e0bdcaa4efebde3e2c4fa47615d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:55.793353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bc9e0bdcaa4efebde3e2c4fa47615d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:55.795295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:55.831934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:55.856541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bc9e0bdcaa4efebde3e2c4fa47615d', 'request_id': 'req-23b4ec53-1fe8-4ecc-b754-d945cd953b32'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.870916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bc9e0bdcaa4efebde3e2c4fa47615d', 'request_id': 'req-23b4ec53-1fe8-4ecc-b754-d945cd953b32'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.894178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bc9e0bdcaa4efebde3e2c4fa47615d', 'request_id': 'req-23b4ec53-1fe8-4ecc-b754-d945cd953b32'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.911199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] The token cache is being invalidate because project 86bc9e0bdcaa4efebde3e2c4fa47615d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:58:55.911491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.912179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-23b4ec53-1fe8-4ecc-b754-d945cd953b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:58:55.913798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 997/1989] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:58:55 2022] DELETE /identity/v3/projects/86bc9e0bdcaa4efebde3e2c4fa47615d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:55.919216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da85b0-dd19-4f4e-b39a-c4c973a0072f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:55.920341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da85b0-dd19-4f4e-b39a-c4c973a0072f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:55.920610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da85b0-dd19-4f4e-b39a-c4c973a0072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:55.941799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-00da85b0-dd19-4f4e-b39a-c4c973a0072f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:55.943048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-00da85b0-dd19-4f4e-b39a-c4c973a0072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:56.013693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 993/1990] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:56.023195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f24f36-3a1c-4e6b-af4c-bf557c8b5b7d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.023554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f24f36-3a1c-4e6b-af4c-bf557c8b5b7d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.024489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f24f36-3a1c-4e6b-af4c-bf557c8b5b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.055871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-13f24f36-3a1c-4e6b-af4c-bf557c8b5b7d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:56.057162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-13f24f36-3a1c-4e6b-af4c-bf557c8b5b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:56.066726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 998/1991] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:56.072235 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da85b0-dd19-4f4e-b39a-c4c973a0072f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.078563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:56.083233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da85b0-dd19-4f4e-b39a-c4c973a0072f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.101952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.103092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.103852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.104626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.106133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:58:56.108093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.109751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.131422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 994/1992] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 14:58:56.138696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f24f36-3a1c-4e6b-af4c-bf557c8b5b7d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.151486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f24f36-3a1c-4e6b-af4c-bf557c8b5b7d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.162497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.162497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.162497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.162497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.163093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.163093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.206761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 999/1993] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:56 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:56.211199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.222669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31aa209-0652-4f13-b4d2-12aa2c1fecc6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.300790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.301563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.301831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.302110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.302858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.305433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.338668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 995/1994] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:56 2022] POST /identity/v3/users => generated 282 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:56.342651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.356254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcc088-6485-4257-bb7f-d3f7f5087676 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.364066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.365338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.365576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.365918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.366608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.368479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.380602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1000/1995] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:56.387181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.398073 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a3270-f3ef-4c54-98ac-38946ea94d5e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.409137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6592c-daa2-4135-aa29-dfbbaa2a0d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.409137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6592c-daa2-4135-aa29-dfbbaa2a0d6c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.409137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6592c-daa2-4135-aa29-dfbbaa2a0d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.409137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6592c-daa2-4135-aa29-dfbbaa2a0d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.409669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6592c-daa2-4135-aa29-dfbbaa2a0d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.411234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6592c-daa2-4135-aa29-dfbbaa2a0d6c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.427309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 996/1996] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:58:56.428871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.439526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204891-75ff-407b-a08e-d78aacbcc153 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.446961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.450972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.450972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.450972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] PATH_INFO: `/v3/projects/d0b3a6b2968a43b4906504cb39b714c8/users/30fbf6984cb8438b893d428e316838b6/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.487100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0b3a6b2968a43b4906504cb39b714c8, user_id=30fbf6984cb8438b893d428e316838b6, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.490332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.507766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1001/1997] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:56 2022] PUT /identity/v3/projects/d0b3a6b2968a43b4906504cb39b714c8/users/30fbf6984cb8438b893d428e316838b6/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:56.524052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd28af-124d-4334-8a0c-bc65dc00f845 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.524606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd28af-124d-4334-8a0c-bc65dc00f845 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.525002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd28af-124d-4334-8a0c-bc65dc00f845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.557578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-22cd28af-124d-4334-8a0c-bc65dc00f845 None None] MFA Rules not processed for user `30fbf6984cb8438b893d428e316838b6`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:56.558861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-22cd28af-124d-4334-8a0c-bc65dc00f845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:56.648939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 997/1998] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:56.655978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.740869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f49ab5-77cd-49ed-9eba-f2e4e19378a9 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.747764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.748451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.748734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.749062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.749764 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.752671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.791436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1002/1999] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:58:56 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:56.795748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd28af-124d-4334-8a0c-bc65dc00f845 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.807474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd28af-124d-4334-8a0c-bc65dc00f845 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.886909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.887656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.888021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.888433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.889283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.891569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.899040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:56.916340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 998/2000] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:58:56 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:58:56.921189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:56.937591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75e9d8-e0cf-4e5c-9ba6-d2974d0b0eb0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:56.951497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:56.952815 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:56.953163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:56.953470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:56.954217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:56.955872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:56.974036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1003/2001] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:58:56.991466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:57.012800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0015e0a-f143-4c25-acac-88031b4f06ec admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:57.017924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:57.018839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:57.019116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:57.019388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] PATH_INFO: `/v3/projects/aca0931a288e42a98301661511910a1d/users/baf9a3f8013d430389afbe193fda10f1/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:57.046551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca0931a288e42a98301661511910a1d, user_id=baf9a3f8013d430389afbe193fda10f1, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:57.049345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:57.065614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 999/2002] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:56 2022] PUT /identity/v3/projects/aca0931a288e42a98301661511910a1d/users/baf9a3f8013d430389afbe193fda10f1/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:57.069947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:57.094626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35d151-cfdb-4f26-b95e-18fdec3676b0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:57.120763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:58:57.167461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52e8e6-3978-4df6-a9cb-66de1498f700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:57.168188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52e8e6-3978-4df6-a9cb-66de1498f700 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:57.168489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52e8e6-3978-4df6-a9cb-66de1498f700 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:57.168872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52e8e6-3978-4df6-a9cb-66de1498f700 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:57.169691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52e8e6-3978-4df6-a9cb-66de1498f700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:57.171452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52e8e6-3978-4df6-a9cb-66de1498f700 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:57.187142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1004/2003] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:58:57.194864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:57.210865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348be37c-4d64-47fc-8ebb-e44918ac64dc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:57.215799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:57.216811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:57.217200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:57.217555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] PATH_INFO: `/v3/projects/aca0931a288e42a98301661511910a1d/users/baf9a3f8013d430389afbe193fda10f1/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:57.219677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca0931a288e42a98301661511910a1d, user_id=baf9a3f8013d430389afbe193fda10f1, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:57.222451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:57.235646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1000/2004] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:58:57 2022] PUT /identity/v3/projects/aca0931a288e42a98301661511910a1d/users/baf9a3f8013d430389afbe193fda10f1/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:58:57.256008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6b77-3af3-42d4-a048-9f09ea133d07 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:57.256344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6b77-3af3-42d4-a048-9f09ea133d07 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:57.256742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b6b77-3af3-42d4-a048-9f09ea133d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:57.288586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-540b6b77-3af3-42d4-a048-9f09ea133d07 None None] MFA Rules not processed for user `baf9a3f8013d430389afbe193fda10f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:58:57.290659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-540b6b77-3af3-42d4-a048-9f09ea133d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:57.443090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1005/2005] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:58:57.455888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:57.457661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:58:57.539389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e0980-a8c6-441d-8607-92c410615140 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:57.543126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:57.543804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:57.544115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:57.544393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:57.559743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:57.561695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:57.577330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1001/2006] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:58:57.664941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6b77-3af3-42d4-a048-9f09ea133d07 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:58:57.678270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b6b77-3af3-42d4-a048-9f09ea133d07 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:58:57.683381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d40dc-9985-4081-b3d9-7e16c04c7f14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:58:57.684070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d40dc-9985-4081-b3d9-7e16c04c7f14 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:58:57.684472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d40dc-9985-4081-b3d9-7e16c04c7f14 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:58:57.684801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d40dc-9985-4081-b3d9-7e16c04c7f14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:58:57.778407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d40dc-9985-4081-b3d9-7e16c04c7f14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:58:57.780902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d40dc-9985-4081-b3d9-7e16c04c7f14 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:58:57.791467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1006/2007] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:58:57 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:00.699205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:00.767335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef98903-f74d-420e-92b0-168ad9a1ba5a service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:00.772027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:00.773013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:00.773810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:00.774086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] PATH_INFO: `/v3/users/30fbf6984cb8438b893d428e316838b6` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:00.775338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30fbf6984cb8438b893d428e316838b6)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:00.777297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:00.836568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30fbf6984cb8438b893d428e316838b6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:00.837215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30fbf6984cb8438b893d428e316838b6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:00.844684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30fbf6984cb8438b893d428e316838b6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:00.845496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30fbf6984cb8438b893d428e316838b6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:00.847248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:00.855197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30fbf6984cb8438b893d428e316838b6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:00.874724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1002/2008] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:00 2022] DELETE /identity/v3/users/30fbf6984cb8438b893d428e316838b6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:00.885503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:00.886068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:00.886068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:00.908717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:00.910030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:00.956287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:00.979189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1007/2009] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:00.988056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:00.989522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:01.054825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172152fc-a9ea-4663-970d-a56a19cf1df0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:01.058938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:01.059610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:01.059897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:01.060173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:01.071285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:01.072884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:01.083486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1003/2010] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:01.479183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:01.482160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:01.505456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da5d37-756c-43b3-9e81-8d764bd6b71e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:01.512816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:01.513706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:01.513996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:01.514283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] PATH_INFO: `/v3/projects/d0b3a6b2968a43b4906504cb39b714c8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:01.515523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0b3a6b2968a43b4906504cb39b714c8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:01.517576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:01.567509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0b3a6b2968a43b4906504cb39b714c8', 'request_id': 'req-7eeba284-acb5-4306-8ae6-8e26c9069922'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.574068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0b3a6b2968a43b4906504cb39b714c8', 'request_id': 'req-7eeba284-acb5-4306-8ae6-8e26c9069922'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.582183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0b3a6b2968a43b4906504cb39b714c8', 'request_id': 'req-7eeba284-acb5-4306-8ae6-8e26c9069922'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.594423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] The token cache is being invalidate because project d0b3a6b2968a43b4906504cb39b714c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:01.594717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.595559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.597266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1008/2011] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:01 2022] DELETE /identity/v3/projects/d0b3a6b2968a43b4906504cb39b714c8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:01.601720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:01.611923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ae5c6-913b-4ad2-8af5-406fb0d7f9e2 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:01.694497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:01.695419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:01.695704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:01.695987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] PATH_INFO: `/v3/users/baf9a3f8013d430389afbe193fda10f1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:01.697277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf9a3f8013d430389afbe193fda10f1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:01.699252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:01.761504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf9a3f8013d430389afbe193fda10f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.762479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf9a3f8013d430389afbe193fda10f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.779672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf9a3f8013d430389afbe193fda10f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.780479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf9a3f8013d430389afbe193fda10f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.789387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1a10490e-62e9-41b2-acae-c7ca8c56a05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf9a3f8013d430389afbe193fda10f1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:01.803857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:01.814134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1004/2012] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:01 2022] DELETE /identity/v3/users/baf9a3f8013d430389afbe193fda10f1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:02.117158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.118632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:02.132335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba284-acb5-4306-8ae6-8e26c9069922 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:02.211532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:02.212410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.212693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.213004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] PATH_INFO: `/v3/projects/aca0931a288e42a98301661511910a1d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.214230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca0931a288e42a98301661511910a1d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:02.216175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:02.265358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca0931a288e42a98301661511910a1d', 'request_id': 'req-f43fc690-e101-465d-9d65-9608f422435d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:02.271826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca0931a288e42a98301661511910a1d', 'request_id': 'req-f43fc690-e101-465d-9d65-9608f422435d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:02.282654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca0931a288e42a98301661511910a1d', 'request_id': 'req-f43fc690-e101-465d-9d65-9608f422435d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:02.294016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:02.298391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] The token cache is being invalidate because project aca0931a288e42a98301661511910a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:02.298898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:02.300116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f43fc690-e101-465d-9d65-9608f422435d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:02.301886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1009/2013] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:02 2022] DELETE /identity/v3/projects/aca0931a288e42a98301661511910a1d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:02.308057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78183d29-7ffc-4bd5-a735-5d21a092f2fa None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.308569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78183d29-7ffc-4bd5-a735-5d21a092f2fa None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.309112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78183d29-7ffc-4bd5-a735-5d21a092f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.329390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-78183d29-7ffc-4bd5-a735-5d21a092f2fa None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:02.330627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-78183d29-7ffc-4bd5-a735-5d21a092f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:02.407403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1005/2014] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:02.415441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ba3e-ade9-4051-afd0-ba2840934e76 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.415737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ba3e-ade9-4051-afd0-ba2840934e76 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.416020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ba3e-ade9-4051-afd0-ba2840934e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.435218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-cc57ba3e-ade9-4051-afd0-ba2840934e76 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:02.436447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cc57ba3e-ade9-4051-afd0-ba2840934e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:02.444534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1010/2015] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:02.449667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78183d29-7ffc-4bd5-a735-5d21a092f2fa None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.460256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78183d29-7ffc-4bd5-a735-5d21a092f2fa None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:02.465720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:02.466434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.466723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.467012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.467767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:02.468089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:02.469752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:02.483823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1006/2016] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:02.488746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ba3e-ade9-4051-afd0-ba2840934e76 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.498608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ba3e-ade9-4051-afd0-ba2840934e76 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:02.503836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:02.504605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.504978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.505213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.505915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:02.508392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:02.547459 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1011/2017] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:02 2022] POST /identity/v3/projects => generated 359 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:02.548994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.559433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b186779-19aa-49b1-90b2-df829d93b38b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:02.621701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:02.622404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.622689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.622971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.623692 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:02.625705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:02.660383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1007/2018] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:02 2022] POST /identity/v3/users => generated 280 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:02.665063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.676633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfae56-6700-4c5b-a02d-fa003a6409b8 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:02.681666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:02.682350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.682641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.682922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.683663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:02.685328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:02.698366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1012/2019] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:02.704531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.718870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79430cda-ed9e-4b97-838e-3c72b0c67aaf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:02.723513 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bcb4a-f391-4af8-821d-a97c83828cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:02.724203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bcb4a-f391-4af8-821d-a97c83828cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.724486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bcb4a-f391-4af8-821d-a97c83828cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.724762 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bcb4a-f391-4af8-821d-a97c83828cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.725783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bcb4a-f391-4af8-821d-a97c83828cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:02.727430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bcb4a-f391-4af8-821d-a97c83828cb5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:02.739126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1008/2020] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:02.743387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.758139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803a94b-ef3f-480e-9043-8d24eb31ebbd admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:02.763934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:02.764876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.765394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.765394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] PATH_INFO: `/v3/projects/33be2a2e61a54f6ea98e8573fb94c686/users/003e89d252274408b36e767669d913f2/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.782893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33be2a2e61a54f6ea98e8573fb94c686, user_id=003e89d252274408b36e767669d913f2, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:02.785843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:02.801292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1013/2021] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:02 2022] PUT /identity/v3/projects/33be2a2e61a54f6ea98e8573fb94c686/users/003e89d252274408b36e767669d913f2/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:02.812879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9e850-c127-4e1e-890d-699c909122ca None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:02.813275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9e850-c127-4e1e-890d-699c909122ca None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:02.813629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9e850-c127-4e1e-890d-699c909122ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:02.844597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-92c9e850-c127-4e1e-890d-699c909122ca None None] MFA Rules not processed for user `003e89d252274408b36e767669d913f2`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:02.847632 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-92c9e850-c127-4e1e-890d-699c909122ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:02.901658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:02.928774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1009/2022] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:02.961820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:02.963526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:03.028723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508bc88-5a3f-4868-b476-d8ce5fd01220 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:03.032579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e32de-781a-4826-a4c2-a7e3f3b1ed9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:03.033632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e32de-781a-4826-a4c2-a7e3f3b1ed9b service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:03.034100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e32de-781a-4826-a4c2-a7e3f3b1ed9b service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:03.034588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e32de-781a-4826-a4c2-a7e3f3b1ed9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:03.046674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e32de-781a-4826-a4c2-a7e3f3b1ed9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:03.048146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e32de-781a-4826-a4c2-a7e3f3b1ed9b service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:03.061444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1014/2023] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:02 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:03.078149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9e850-c127-4e1e-890d-699c909122ca None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:03.157094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9e850-c127-4e1e-890d-699c909122ca None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:03.162697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:03.164367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:03.164653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:03.165011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] PATH_INFO: `/v3/users/003e89d252274408b36e767669d913f2` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:03.166284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=003e89d252274408b36e767669d913f2)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:03.168307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:03.232167 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003e89d252274408b36e767669d913f2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:03.232961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003e89d252274408b36e767669d913f2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:03.245110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003e89d252274408b36e767669d913f2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:03.253925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003e89d252274408b36e767669d913f2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:03.262890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003e89d252274408b36e767669d913f2'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:03.282332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1010/2024] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:03 2022] DELETE /identity/v3/users/003e89d252274408b36e767669d913f2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:03.287213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:03.287762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:03.287762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:03.309237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:03.310453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:03.393570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1015/2025] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:03.409638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:03.410020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:03.512387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbbdb9-fb2e-4962-b822-73d458383afd admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:03.516279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14730-c8a7-489f-9409-d3e5bcd4153b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:03.517096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14730-c8a7-489f-9409-d3e5bcd4153b service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:03.517274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14730-c8a7-489f-9409-d3e5bcd4153b service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:03.517562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14730-c8a7-489f-9409-d3e5bcd4153b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:03.529643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14730-c8a7-489f-9409-d3e5bcd4153b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:03.531105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14730-c8a7-489f-9409-d3e5bcd4153b service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:03.544710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1011/2026] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:04.114274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:04.114274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:04.133362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2438a-6bb6-469c-9e7e-ad114827ad84 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:04.146125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:04.146911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.147093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.147351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] PATH_INFO: `/v3/projects/33be2a2e61a54f6ea98e8573fb94c686` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.169905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33be2a2e61a54f6ea98e8573fb94c686)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:04.169905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:04.222478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:04.236983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33be2a2e61a54f6ea98e8573fb94c686', 'request_id': 'req-365ece6c-65ac-4721-ac98-ef5009e602c1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:04.244136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33be2a2e61a54f6ea98e8573fb94c686', 'request_id': 'req-365ece6c-65ac-4721-ac98-ef5009e602c1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:04.253711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33be2a2e61a54f6ea98e8573fb94c686', 'request_id': 'req-365ece6c-65ac-4721-ac98-ef5009e602c1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:04.271445 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] The token cache is being invalidate because project 33be2a2e61a54f6ea98e8573fb94c686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:04.271953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:04.272767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-365ece6c-65ac-4721-ac98-ef5009e602c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:04.275162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1016/2027] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:04 2022] DELETE /identity/v3/projects/33be2a2e61a54f6ea98e8573fb94c686 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:04.280569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02d05-0d25-4056-bb87-d62e850025f3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.281197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02d05-0d25-4056-bb87-d62e850025f3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.281197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02d05-0d25-4056-bb87-d62e850025f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.300481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-48a02d05-0d25-4056-bb87-d62e850025f3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:04.301751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-48a02d05-0d25-4056-bb87-d62e850025f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:04.313953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:04.372515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1012/2028] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:04.382707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67752cfb-dd26-4f65-8619-31dd37a41b3f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.383763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67752cfb-dd26-4f65-8619-31dd37a41b3f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.383966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67752cfb-dd26-4f65-8619-31dd37a41b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.404227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-67752cfb-dd26-4f65-8619-31dd37a41b3f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:04.405508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-67752cfb-dd26-4f65-8619-31dd37a41b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:04.427510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1017/2029] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:04.430375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02d05-0d25-4056-bb87-d62e850025f3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:04.449383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02d05-0d25-4056-bb87-d62e850025f3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:04.464496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:04.465548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.465994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.466266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.467149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:04.467689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:04.472083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:04.492919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1013/2030] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:04.497278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67752cfb-dd26-4f65-8619-31dd37a41b3f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:04.509529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67752cfb-dd26-4f65-8619-31dd37a41b3f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:04.517033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:04.517982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.518449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.518880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.519783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:04.521840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:04.555472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1018/2031] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:04 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:59:04.560026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:04.571012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e720d-f65a-4c68-99a8-0d099f004c6a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:04.634354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:04.635035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.635319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.635595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.636288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:04.638266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:04.660742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1014/2032] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:04 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:04.665585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:04.678624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e65d-9a1d-4dc8-90c3-687d6ed82205 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:04.684096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:04.684809 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.685133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.685415 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.686158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:04.687882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:04.707382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1019/2033] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:04.708530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:04.720371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35956383-626e-4c07-995a-3d51f587182a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:04.725238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfd240-a028-4b02-b8d4-e6aff5af7b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:04.725918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfd240-a028-4b02-b8d4-e6aff5af7b99 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.726223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfd240-a028-4b02-b8d4-e6aff5af7b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.727025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfd240-a028-4b02-b8d4-e6aff5af7b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.727931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfd240-a028-4b02-b8d4-e6aff5af7b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:04.729509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfd240-a028-4b02-b8d4-e6aff5af7b99 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:04.743099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1015/2034] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:04.749472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:04.765357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79018cdc-38d5-4b44-a071-a076d27cd59d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:04.772280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:04.773357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.774165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.774165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] PATH_INFO: `/v3/projects/8d34021f0bea45bf9226c74eccf002cc/users/27de8b18d5a14425879ad7ec60002bb7/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.792716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d34021f0bea45bf9226c74eccf002cc, user_id=27de8b18d5a14425879ad7ec60002bb7, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:04.795475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:04.811851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1020/2035] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:04 2022] PUT /identity/v3/projects/8d34021f0bea45bf9226c74eccf002cc/users/27de8b18d5a14425879ad7ec60002bb7/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:04.827760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219986-200a-4d3f-8d69-193e3076fe43 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:04.828048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219986-200a-4d3f-8d69-193e3076fe43 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:04.828327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219986-200a-4d3f-8d69-193e3076fe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:04.865550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8c219986-200a-4d3f-8d69-193e3076fe43 None None] MFA Rules not processed for user `27de8b18d5a14425879ad7ec60002bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:04.866855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8c219986-200a-4d3f-8d69-193e3076fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:04.964609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1016/2036] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:04.969307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.041670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbbc5-dbc5-4929-ad00-e66eaca3e41c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.046460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.047134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.047429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.047710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.048425 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.050423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.087991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1021/2037] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:04 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:05.092141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c219986-200a-4d3f-8d69-193e3076fe43 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.103725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c219986-200a-4d3f-8d69-193e3076fe43 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.115859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:05.181506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.182201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.182483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.182850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.183632 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.185630 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.212472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1017/2038] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:05 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:05.217721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.228373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056730-2a49-4f57-b011-c7ee711e3eeb admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.234311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.235068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.235356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.235628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.236437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.238347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.253411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1022/2039] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:05.254882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.266218 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec2321-52c2-451f-a3f8-8dcbee797cab admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.274157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273680f-8330-49b6-91e1-611d86a4d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.275351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273680f-8330-49b6-91e1-611d86a4d565 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.276801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273680f-8330-49b6-91e1-611d86a4d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.276801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273680f-8330-49b6-91e1-611d86a4d565 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.278105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273680f-8330-49b6-91e1-611d86a4d565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.282395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273680f-8330-49b6-91e1-611d86a4d565 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.308489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1018/2040] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:05.310247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.326605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41894d-7d0a-4675-97bb-e2daf8ec519c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.332599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.333756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.334765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.334765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] PATH_INFO: `/v3/projects/5c6d0ed71ffe4cdebe22422c3f8c31fe/users/00f07f65d54f473296b4043b1aa20a7b/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.360473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6d0ed71ffe4cdebe22422c3f8c31fe, user_id=00f07f65d54f473296b4043b1aa20a7b, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.363386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.381060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1023/2041] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:05 2022] PUT /identity/v3/projects/5c6d0ed71ffe4cdebe22422c3f8c31fe/users/00f07f65d54f473296b4043b1aa20a7b/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:05.399260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac2681-0a8d-4155-8504-779ea282bb20 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.399829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac2681-0a8d-4155-8504-779ea282bb20 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.400262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac2681-0a8d-4155-8504-779ea282bb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.440528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-25ac2681-0a8d-4155-8504-779ea282bb20 None None] MFA Rules not processed for user `00f07f65d54f473296b4043b1aa20a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:05.442166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-25ac2681-0a8d-4155-8504-779ea282bb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:05.526657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1019/2042] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:05.532332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.546786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:05.610940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a814b-869f-45ab-9395-770cd570a27b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.615885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.616725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.617517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.617857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.618509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.620458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.651540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1024/2043] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:05 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:05.653057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac2681-0a8d-4155-8504-779ea282bb20 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.666239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac2681-0a8d-4155-8504-779ea282bb20 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.753419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.754125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.754493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.754727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.755424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.757470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.789707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1020/2044] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:05 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:05.791776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.808053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48531f-57cd-4294-89ba-8c34f4c5eff9 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.817783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.818496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.818783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.819062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.819817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.821716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.839392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1025/2045] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:05.843749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.855107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2a3e-1646-473c-9639-7e9790a072b9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:05.860059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:05.861001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:05.861287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:05.861560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] PATH_INFO: `/v3/projects/93eb37a6e245478584add425681a9068/users/73b7501752ba4d94a0682bb6ba3432be/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:05.864137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:05.918394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eb37a6e245478584add425681a9068, user_id=73b7501752ba4d94a0682bb6ba3432be, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:05.925202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:05.963413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1021/2046] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:05 2022] PUT /identity/v3/projects/93eb37a6e245478584add425681a9068/users/73b7501752ba4d94a0682bb6ba3432be/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:05.968787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:05.997094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948181f5-3f97-4669-b8a9-c5e3b61f0ba0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:06.136961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5acc73-4226-4569-ae10-f892ee5d5c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:06.137662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5acc73-4226-4569-ae10-f892ee5d5c86 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:06.137954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5acc73-4226-4569-ae10-f892ee5d5c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:06.138238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5acc73-4226-4569-ae10-f892ee5d5c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:06.138976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5acc73-4226-4569-ae10-f892ee5d5c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:06.140610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5acc73-4226-4569-ae10-f892ee5d5c86 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:06.153045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1026/2047] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:06.157250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:06.169072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62869-ad2b-4bd5-a3a9-29de2d96ef3e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:06.178931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:06.180559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:06.180856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:06.181162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] PATH_INFO: `/v3/projects/93eb37a6e245478584add425681a9068/users/73b7501752ba4d94a0682bb6ba3432be/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:06.183192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eb37a6e245478584add425681a9068, user_id=73b7501752ba4d94a0682bb6ba3432be, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:06.185900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:06.200907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1022/2048] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:06 2022] PUT /identity/v3/projects/93eb37a6e245478584add425681a9068/users/73b7501752ba4d94a0682bb6ba3432be/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:59:06.212170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e9ee-1efe-4a7e-a9e3-7889daa5b8b3 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:06.212462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e9ee-1efe-4a7e-a9e3-7889daa5b8b3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:06.212747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062e9ee-1efe-4a7e-a9e3-7889daa5b8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:06.243260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5062e9ee-1efe-4a7e-a9e3-7889daa5b8b3 None None] MFA Rules not processed for user `73b7501752ba4d94a0682bb6ba3432be`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:06.244504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5062e9ee-1efe-4a7e-a9e3-7889daa5b8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:06.337823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1027/2049] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:06.346682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:06.348735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:06.412506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeb6eb-58df-4ece-9948-5f153b3afa0d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:06.417638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:06.418054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:06.418420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:06.418621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:06.487174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:06.488691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:06.499843 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1023/2050] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:09.150603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062e9ee-1efe-4a7e-a9e3-7889daa5b8b3 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:09.157998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:09.188863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062e9ee-1efe-4a7e-a9e3-7889daa5b8b3 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:09.192875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:09.193597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:09.193994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:09.194288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:09.294595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:09.296263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:09.308374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1028/2051] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:59:10.208903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:10.241104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655b44d-b994-47a5-b1e9-ef657d64456b service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:10.245679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dde4f8-358c-4413-979a-bb36154dc737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:10.246412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde4f8-358c-4413-979a-bb36154dc737 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:10.246764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde4f8-358c-4413-979a-bb36154dc737 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:10.247054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dde4f8-358c-4413-979a-bb36154dc737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:10.262509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dde4f8-358c-4413-979a-bb36154dc737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:10.263995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dde4f8-358c-4413-979a-bb36154dc737 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:10.279626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1024/2052] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:59:15.469214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:15.538531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298ad27-ad73-4e30-ab12-20e72f2cbc4a service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:15.543726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:15.544529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:15.544814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:15.545126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] PATH_INFO: `/v3/users/27de8b18d5a14425879ad7ec60002bb7` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:15.546343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=27de8b18d5a14425879ad7ec60002bb7)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:15.548318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:15.615750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27de8b18d5a14425879ad7ec60002bb7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:15.616489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27de8b18d5a14425879ad7ec60002bb7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:15.625862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27de8b18d5a14425879ad7ec60002bb7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:15.625862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27de8b18d5a14425879ad7ec60002bb7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:15.643747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27de8b18d5a14425879ad7ec60002bb7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:15.669005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1029/2053] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:15 2022] DELETE /identity/v3/users/27de8b18d5a14425879ad7ec60002bb7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:15.674603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:15.675535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:15.675971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:15.698334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:15.699880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:15.781379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1025/2054] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:15.793041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:15.794518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:15.817530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:15.870299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112cc46-c471-4eeb-9170-cdb52b1d5cdc admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:15.874549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:15.875416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:15.875695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:15.875966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:15.889196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:15.890789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:15.906026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1030/2055] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:16.317365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:16.318869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:16.329146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:16.331823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad9e48-c395-46b7-9788-11b64d9c2bc5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:16.341046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:16.342945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:16.342945 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:16.343353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] PATH_INFO: `/v3/projects/8d34021f0bea45bf9226c74eccf002cc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:16.345963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d34021f0bea45bf9226c74eccf002cc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:16.347320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:16.411997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d34021f0bea45bf9226c74eccf002cc', 'request_id': 'req-769e5f85-544a-4e4e-958f-813bbfee8a54'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.418768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d34021f0bea45bf9226c74eccf002cc', 'request_id': 'req-769e5f85-544a-4e4e-958f-813bbfee8a54'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.427132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d34021f0bea45bf9226c74eccf002cc', 'request_id': 'req-769e5f85-544a-4e4e-958f-813bbfee8a54'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.444345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] The token cache is being invalidate because project 8d34021f0bea45bf9226c74eccf002cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:16.444765 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.445532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.447169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1026/2056] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:16 2022] DELETE /identity/v3/projects/8d34021f0bea45bf9226c74eccf002cc => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:16.451612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:16.461779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573eb84c-21db-4638-951b-e133889fb815 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:16.537239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:16.539097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:16.539557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:16.539954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] PATH_INFO: `/v3/users/00f07f65d54f473296b4043b1aa20a7b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:16.541380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f07f65d54f473296b4043b1aa20a7b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:16.543724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:16.616993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f07f65d54f473296b4043b1aa20a7b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.617747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f07f65d54f473296b4043b1aa20a7b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.618383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f07f65d54f473296b4043b1aa20a7b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.625883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f07f65d54f473296b4043b1aa20a7b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.640474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f07f65d54f473296b4043b1aa20a7b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:16.661141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1031/2057] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:16 2022] DELETE /identity/v3/users/00f07f65d54f473296b4043b1aa20a7b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:16.970734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:16.972376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:16.992217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e5f85-544a-4e4e-958f-813bbfee8a54 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:17.067926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:17.068881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:17.069170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:17.069446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] PATH_INFO: `/v3/projects/5c6d0ed71ffe4cdebe22422c3f8c31fe` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:17.071282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6d0ed71ffe4cdebe22422c3f8c31fe)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:17.073274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:17.125681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6d0ed71ffe4cdebe22422c3f8c31fe', 'request_id': 'req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.131079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6d0ed71ffe4cdebe22422c3f8c31fe', 'request_id': 'req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.139684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6d0ed71ffe4cdebe22422c3f8c31fe', 'request_id': 'req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.152182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] The token cache is being invalidate because project 5c6d0ed71ffe4cdebe22422c3f8c31fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:17.153310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.153310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.156215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1027/2058] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:16 2022] DELETE /identity/v3/projects/5c6d0ed71ffe4cdebe22422c3f8c31fe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:17.159464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:17.161240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:17.176986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b654fe1-775e-4e2d-8a7c-f8a9dae03c14 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:17.223237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:17.256865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:17.258466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:17.258846 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:17.259594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] PATH_INFO: `/v3/users/73b7501752ba4d94a0682bb6ba3432be` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:17.260816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73b7501752ba4d94a0682bb6ba3432be)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:17.262912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:17.323490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73b7501752ba4d94a0682bb6ba3432be'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.324242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73b7501752ba4d94a0682bb6ba3432be'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.324905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73b7501752ba4d94a0682bb6ba3432be'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.332636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73b7501752ba4d94a0682bb6ba3432be'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.346247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a7694cb7-80c5-44eb-976b-4aa7efd2e20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73b7501752ba4d94a0682bb6ba3432be'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.360009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1032/2059] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:17 2022] DELETE /identity/v3/users/73b7501752ba4d94a0682bb6ba3432be => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:17.690823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:17.692511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:17.710659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50836f87-d9ee-4bd2-8f91-7ace05f1e79b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:17.731848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:17.797568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:17.798594 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:17.798924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:17.799936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] PATH_INFO: `/v3/projects/93eb37a6e245478584add425681a9068` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:17.801183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93eb37a6e245478584add425681a9068)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:17.803413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:17.867579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93eb37a6e245478584add425681a9068', 'request_id': 'req-46a2836e-d0ef-49b6-a039-46c576b62135'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.874120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93eb37a6e245478584add425681a9068', 'request_id': 'req-46a2836e-d0ef-49b6-a039-46c576b62135'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.882269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93eb37a6e245478584add425681a9068', 'request_id': 'req-46a2836e-d0ef-49b6-a039-46c576b62135'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.896395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] The token cache is being invalidate because project 93eb37a6e245478584add425681a9068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:17.896683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.897380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-46a2836e-d0ef-49b6-a039-46c576b62135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:17.899704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1028/2060] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:17 2022] DELETE /identity/v3/projects/93eb37a6e245478584add425681a9068 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:17.904826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99472833-e4fa-409a-b6f4-2f307447f90a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:17.905693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99472833-e4fa-409a-b6f4-2f307447f90a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:17.905693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99472833-e4fa-409a-b6f4-2f307447f90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:17.925518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-99472833-e4fa-409a-b6f4-2f307447f90a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:17.926773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-99472833-e4fa-409a-b6f4-2f307447f90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:18.004690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1033/2061] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:18.013929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1c75-f399-4a12-b95a-5ac0c8c16148 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.014464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1c75-f399-4a12-b95a-5ac0c8c16148 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.014584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1c75-f399-4a12-b95a-5ac0c8c16148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.038000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5b9f1c75-f399-4a12-b95a-5ac0c8c16148 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:18.039901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5b9f1c75-f399-4a12-b95a-5ac0c8c16148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:18.048479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1029/2062] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:18.054369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99472833-e4fa-409a-b6f4-2f307447f90a None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:18.065900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99472833-e4fa-409a-b6f4-2f307447f90a None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:18.071044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:18.071789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.072073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.072444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.073224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:18.073613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:18.075294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:18.090697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1034/2063] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:18.095586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f1c75-f399-4a12-b95a-5ac0c8c16148 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:18.111608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f1c75-f399-4a12-b95a-5ac0c8c16148 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:18.117196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:18.118381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.118567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.118906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.119692 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:18.121976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:18.167434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1030/2064] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:18 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:18.172145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:18.186650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf72f63f-fc4b-4ca1-8044-f0a9db10c445 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:18.240606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:18.260701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:18.261424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.261714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.261994 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.262903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:18.264888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:18.287420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1035/2065] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:18 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:18.291727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:18.302990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91bf4-fbea-4373-b0f0-31539deec2ed admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:18.309690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:18.310511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.310842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.311162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.311910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:18.313735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:18.330364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1031/2066] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:18.336085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:18.352463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18684668-5307-4875-8646-8bef2c506872 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:18.358823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcde19d-3e52-4b47-87b0-9012328c265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:18.359508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcde19d-3e52-4b47-87b0-9012328c265c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.359799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcde19d-3e52-4b47-87b0-9012328c265c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.360082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcde19d-3e52-4b47-87b0-9012328c265c admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.361312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcde19d-3e52-4b47-87b0-9012328c265c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:18.362861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcde19d-3e52-4b47-87b0-9012328c265c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:18.375666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1036/2067] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:18.380596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:18.397207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98245a76-c991-4491-a9bc-4e3e81f2bbdb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:18.407767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:18.408695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.409016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.409294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] PATH_INFO: `/v3/projects/9da94b7d625146c694e6d96613699f2e/users/97fea71b89024b049438af5f73a28165/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.427856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9da94b7d625146c694e6d96613699f2e, user_id=97fea71b89024b049438af5f73a28165, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:18.430983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:18.462698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1032/2068] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:18 2022] PUT /identity/v3/projects/9da94b7d625146c694e6d96613699f2e/users/97fea71b89024b049438af5f73a28165/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:18.476443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ff127-6900-400b-a3c4-69e5fe047543 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.476733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ff127-6900-400b-a3c4-69e5fe047543 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.477043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ff127-6900-400b-a3c4-69e5fe047543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.515096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-cf3ff127-6900-400b-a3c4-69e5fe047543 None None] MFA Rules not processed for user `97fea71b89024b049438af5f73a28165`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:18.516519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cf3ff127-6900-400b-a3c4-69e5fe047543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:18.653154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1037/2069] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:18.666127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:18.668095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:18.805326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:18.883396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c32da-d1bd-4d14-9442-1893d09725ae admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:18.887417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30faca0-e1a6-42e4-91ea-fc0e80c5ed52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:18.888195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30faca0-e1a6-42e4-91ea-fc0e80c5ed52 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:18.888480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30faca0-e1a6-42e4-91ea-fc0e80c5ed52 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:18.888763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30faca0-e1a6-42e4-91ea-fc0e80c5ed52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:18.899181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30faca0-e1a6-42e4-91ea-fc0e80c5ed52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:18.900731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30faca0-e1a6-42e4-91ea-fc0e80c5ed52 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:18.918711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1033/2070] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:22.663136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ff127-6900-400b-a3c4-69e5fe047543 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:22.754554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ff127-6900-400b-a3c4-69e5fe047543 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:22.762477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:22.763786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:22.764201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:22.764647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] PATH_INFO: `/v3/users/97fea71b89024b049438af5f73a28165` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:22.767034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fea71b89024b049438af5f73a28165)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:22.770748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:22.840582 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fea71b89024b049438af5f73a28165'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:22.841296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fea71b89024b049438af5f73a28165'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:22.841923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fea71b89024b049438af5f73a28165'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:22.847538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:22.852631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fea71b89024b049438af5f73a28165'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:22.867475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fea71b89024b049438af5f73a28165'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:22.884208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1038/2071] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:22 2022] DELETE /identity/v3/users/97fea71b89024b049438af5f73a28165 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:22.888893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:22.889356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:22.889792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:22.909494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:22.910896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:22.978690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1034/2072] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:22.989142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:22.990707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:23.087525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928a53c-d6b1-4c7f-988b-b09d75c3c1e4 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:23.091678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de81ae5-59c8-465c-a351-7923d89816ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:23.092339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de81ae5-59c8-465c-a351-7923d89816ba service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:23.092656 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de81ae5-59c8-465c-a351-7923d89816ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:23.093324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de81ae5-59c8-465c-a351-7923d89816ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:23.115676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de81ae5-59c8-465c-a351-7923d89816ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:23.115676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de81ae5-59c8-465c-a351-7923d89816ba service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:23.127504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1039/2073] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:23.460183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:23.462092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:23.475962 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb7245-8248-43db-b113-dd4398c6eef1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:23.481644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:23.482631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:23.482925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:23.483186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] PATH_INFO: `/v3/users/3e207b9592764083975305553b948172` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:23.488374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e207b9592764083975305553b948172)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:23.491116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:23.569135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e207b9592764083975305553b948172'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:23.571471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e207b9592764083975305553b948172'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:23.583775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e207b9592764083975305553b948172'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:23.584721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e207b9592764083975305553b948172'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:23.606147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e207b9592764083975305553b948172'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:23.638107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1035/2074] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:23 2022] DELETE /identity/v3/users/3e207b9592764083975305553b948172 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:23.640169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:23.640465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:23.640761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:23.663724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:23.665451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:23.674332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:23.675984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:23.772455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1040/2075] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:23.798113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:23.800485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:23.838131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:23.887023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85607-edf2-4ed4-a171-c10ea22fe1df admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:23.891995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:23.892964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:23.893263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:23.893467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] PATH_INFO: `/v3/projects/9da94b7d625146c694e6d96613699f2e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:23.894860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9da94b7d625146c694e6d96613699f2e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:23.896632 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:23.907393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b433ae-a513-42cd-8ab6-e7ea1c881c2e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:23.912172 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e61a5-388e-4e25-8e48-4ae1644ef6ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:23.913068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e61a5-388e-4e25-8e48-4ae1644ef6ac service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:23.913873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e61a5-388e-4e25-8e48-4ae1644ef6ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:23.914255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e61a5-388e-4e25-8e48-4ae1644ef6ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:23.937570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e61a5-388e-4e25-8e48-4ae1644ef6ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:23.939317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e61a5-388e-4e25-8e48-4ae1644ef6ac service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:23.951545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1041/2076] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:23.979030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9da94b7d625146c694e6d96613699f2e', 'request_id': 'req-d2c29804-61a3-498f-b451-61f87d6a97b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:23.986304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9da94b7d625146c694e6d96613699f2e', 'request_id': 'req-d2c29804-61a3-498f-b451-61f87d6a97b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:23.996724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9da94b7d625146c694e6d96613699f2e', 'request_id': 'req-d2c29804-61a3-498f-b451-61f87d6a97b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.022316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] The token cache is being invalidate because project 9da94b7d625146c694e6d96613699f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:24.023603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.025468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d2c29804-61a3-498f-b451-61f87d6a97b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.029367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1036/2077] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:23 2022] DELETE /identity/v3/projects/9da94b7d625146c694e6d96613699f2e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:24.043099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fa49-cc74-4d76-af9a-8203503fa499 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.043467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fa49-cc74-4d76-af9a-8203503fa499 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.043757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fa49-cc74-4d76-af9a-8203503fa499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.066363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ca17fa49-cc74-4d76-af9a-8203503fa499 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:24.067283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ca17fa49-cc74-4d76-af9a-8203503fa499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:24.080156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:24.218636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1042/2078] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:24.226837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e3a81-90d2-4bb6-9bd2-9f13be05547c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.227133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e3a81-90d2-4bb6-9bd2-9f13be05547c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.227409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e3a81-90d2-4bb6-9bd2-9f13be05547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.256370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-770e3a81-90d2-4bb6-9bd2-9f13be05547c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:24.258191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-770e3a81-90d2-4bb6-9bd2-9f13be05547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:24.273768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1037/2079] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:24.275463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fa49-cc74-4d76-af9a-8203503fa499 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:24.292148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fa49-cc74-4d76-af9a-8203503fa499 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:24.303276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:24.304545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.304859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.305146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.305897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:24.306211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:24.308215 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:24.351729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1043/2080] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:24.356217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e3a81-90d2-4bb6-9bd2-9f13be05547c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:24.376617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e3a81-90d2-4bb6-9bd2-9f13be05547c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:24.385003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:24.386068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.386532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.386991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.388694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:24.392215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:24.428702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:24.432455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:24.454013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1038/2081] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:24 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:24.458947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d2178-f95e-4e00-9cc6-635c175a1fc1 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:24.461045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:24.568088 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:24.569150 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.569611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.569997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] PATH_INFO: `/v3/projects/7206ab05cd174bae9d62eebef69af57f` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.571737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7206ab05cd174bae9d62eebef69af57f)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:24.574190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:24.601101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b012e75-ee93-410c-8e40-4086c9095a58 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:24.607647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:24.608541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.609031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.609388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.610196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:24.612866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:24.643669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1039/2082] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:24 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:24.649893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:24.659589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7206ab05cd174bae9d62eebef69af57f', 'request_id': 'req-470d3325-33b7-4636-b419-05fdf4ab3f9b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.674352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f3c90-0cab-4aa2-b3eb-a567016abcbd admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:24.676516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7206ab05cd174bae9d62eebef69af57f', 'request_id': 'req-470d3325-33b7-4636-b419-05fdf4ab3f9b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.687591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7206ab05cd174bae9d62eebef69af57f', 'request_id': 'req-470d3325-33b7-4636-b419-05fdf4ab3f9b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.707483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] The token cache is being invalidate because project 7206ab05cd174bae9d62eebef69af57f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:24.708160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.709076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-470d3325-33b7-4636-b419-05fdf4ab3f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:24.712365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1044/2083] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:24 2022] DELETE /identity/v3/projects/7206ab05cd174bae9d62eebef69af57f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:24.718804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192386f-d9d9-4a65-b865-72e6f70c68a1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.723876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192386f-d9d9-4a65-b865-72e6f70c68a1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.723876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192386f-d9d9-4a65-b865-72e6f70c68a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.740690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:24.778136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8192386f-d9d9-4a65-b865-72e6f70c68a1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:24.783207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8192386f-d9d9-4a65-b865-72e6f70c68a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:24.827462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:24.828480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.829021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.832575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.836116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:24.839295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:24.852414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:24.866054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1040/2084] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:24.870039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:24.873698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:24.891188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4b3bb-29a6-41c6-926d-5b1a3459f12e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:24.900545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:24.902515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.903378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.903874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.905000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:24.906947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:24.920474 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1041/2085] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:24.923161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1045/2086] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:24.925796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:24.931954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802bc17-2262-48b5-90c8-c5ec80db7df8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.932831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802bc17-2262-48b5-90c8-c5ec80db7df8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.933907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802bc17-2262-48b5-90c8-c5ec80db7df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.951999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d3505-de82-410d-81d5-776d0eb54a0f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:24.969498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8802bc17-2262-48b5-90c8-c5ec80db7df8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:24.970751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8802bc17-2262-48b5-90c8-c5ec80db7df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:24.975290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0ef41-cd4d-40a8-9eeb-1bf74cde584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:24.977299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ef41-cd4d-40a8-9eeb-1bf74cde584a admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:24.977299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ef41-cd4d-40a8-9eeb-1bf74cde584a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:24.977445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ef41-cd4d-40a8-9eeb-1bf74cde584a admin admin] PATH_INFO: `/v3/projects/b475cd8351374fb0a0853b4d4534dfd5/users/5b662d94c0f54f26b8283cdc22770df7/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:24.993277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1046/2087] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:24.997842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802bc17-2262-48b5-90c8-c5ec80db7df8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:25.000300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0ef41-cd4d-40a8-9eeb-1bf74cde584a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b475cd8351374fb0a0853b4d4534dfd5, user_id=5b662d94c0f54f26b8283cdc22770df7, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.003638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0ef41-cd4d-40a8-9eeb-1bf74cde584a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.028728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802bc17-2262-48b5-90c8-c5ec80db7df8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:25.042138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1042/2088] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:24 2022] PUT /identity/v3/projects/b475cd8351374fb0a0853b4d4534dfd5/users/5b662d94c0f54f26b8283cdc22770df7/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:25.048402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:25.049135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.049460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.049742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.050438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:25.050816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.053392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.053634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.053788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.053993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.075660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1047/2089] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:25.077138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:25.092193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c28f0-73f8-4a44-9534-d95251719835 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:25.093410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] MFA Rules not processed for user `5b662d94c0f54f26b8283cdc22770df7`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:25.094775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:25.203808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:25.204992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.205511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.206011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.207274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.210905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.238702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1043/2090] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:25.261088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:25.264478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:25.280906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1048/2091] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:25 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:25.286321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:25.409346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8eec0-5621-4985-a2cc-d0d9ac92789e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:25.487507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde869e-fb8b-40f1-a9d9-356e23537d01 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:25.494263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:25.495220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:25.496245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.496611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.496816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.497839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.499784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.507923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:25.508788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.509748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.510048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.537618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1049/2092] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:25 2022] POST /identity/v3/users => generated 288 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:25.545050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:25.558730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e5a20-4411-47b9-a353-58198477bf77 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:25.589602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:25.590413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.590690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.590964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.592046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.600602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.625243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.626991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.641096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1044/2093] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:25.651140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1050/2094] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:25.655278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:25.681361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07f0f3-80e5-40ab-943c-74df13ba8698 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:25.694925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dbe60-757d-4326-8901-afb347b927fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:25.695617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dbe60-757d-4326-8901-afb347b927fc admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.695901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dbe60-757d-4326-8901-afb347b927fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.696191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dbe60-757d-4326-8901-afb347b927fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.697305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dbe60-757d-4326-8901-afb347b927fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.704085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dbe60-757d-4326-8901-afb347b927fc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.732708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1045/2095] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:25.736179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:25.749454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ad752-e0c0-43d1-adc5-34a15a437f5c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:25.754816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:25.755719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.756005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.756285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] PATH_INFO: `/v3/projects/51bee9877ae34a1e95bf97e718159b2c/users/04f698df0a07477e833701ea15139321/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.778871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bee9877ae34a1e95bf97e718159b2c, user_id=04f698df0a07477e833701ea15139321, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:25.787474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:25.791253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:25.807187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1051/2096] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:25 2022] PUT /identity/v3/projects/51bee9877ae34a1e95bf97e718159b2c/users/04f698df0a07477e833701ea15139321/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:25.817921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee244cad-1b78-43c6-8953-7e25827a7799 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:25.818135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee244cad-1b78-43c6-8953-7e25827a7799 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:25.818350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee244cad-1b78-43c6-8953-7e25827a7799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:25.881610 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ee244cad-1b78-43c6-8953-7e25827a7799 None None] MFA Rules not processed for user `04f698df0a07477e833701ea15139321`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:25.883585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ee244cad-1b78-43c6-8953-7e25827a7799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:25.996410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1046/2097] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:26.011497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:26.012593 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:26.149414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147904ad-6059-4646-82d3-c1b36eab0a1c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:26.179423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9bd87-7143-4f70-9d9c-28a5b04a1a9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:26.183333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9bd87-7143-4f70-9d9c-28a5b04a1a9e service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:26.183721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9bd87-7143-4f70-9d9c-28a5b04a1a9e service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:26.184132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9bd87-7143-4f70-9d9c-28a5b04a1a9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:26.213183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9bd87-7143-4f70-9d9c-28a5b04a1a9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:26.217529 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9bd87-7143-4f70-9d9c-28a5b04a1a9e service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:26.248939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1052/2098] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:27.009687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0cfb-859e-4c31-afa4-dc28dc3e0599 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:27.010632 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0cfb-859e-4c31-afa4-dc28dc3e0599 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:27.010832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a0cfb-859e-4c31-afa4-dc28dc3e0599 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:27.015861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1047/2099] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 14:59:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 14:59:27.030391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:27.030966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:27.031062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:27.110869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:27.112068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:27.200640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1053/2100] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:27.349717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0cfb-859e-4c31-afa4-dc28dc3e0599 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:27.352543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9f6a0cfb-859e-4c31-afa4-dc28dc3e0599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:27.371875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:27.377580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:27.383788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a0cfb-859e-4c31-afa4-dc28dc3e0599 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:27.392390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:27.393462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:27.393904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:27.394332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:27.408766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:27.410395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:27.425539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1048/2101] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:27.487741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:27.504659 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4f555-062d-486b-90fa-10cf6e9fc4d9 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:27.509457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:27.510349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:27.511362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:27.511362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] PATH_INFO: `/v3/users/1c246b5f70c54d678d827ea43d5c7a38` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:27.512368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c246b5f70c54d678d827ea43d5c7a38)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:27.514531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:27.569116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c246b5f70c54d678d827ea43d5c7a38'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:27.569936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c246b5f70c54d678d827ea43d5c7a38'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:27.570575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c246b5f70c54d678d827ea43d5c7a38'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:27.577819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c246b5f70c54d678d827ea43d5c7a38'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:27.603748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9351557-dec3-479e-9883-7c008cbf5126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c246b5f70c54d678d827ea43d5c7a38'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:27.619485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1054/2102] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:27 2022] DELETE /identity/v3/users/1c246b5f70c54d678d827ea43d5c7a38 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 14:59:27.830137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:27.853731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34873f-9728-495c-850d-2d95fcc3d47c service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:27.940248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:27.941155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:27.941436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:27.941708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] PATH_INFO: `/v3/projects/a0788d395b4e4632b29e263608e28973` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:27.943046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0788d395b4e4632b29e263608e28973)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:27.945533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:27.952592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:27.999778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0788d395b4e4632b29e263608e28973', 'request_id': 'req-5362f28c-ff89-4534-805c-2e52dc4cf345'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:28.006265 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0788d395b4e4632b29e263608e28973', 'request_id': 'req-5362f28c-ff89-4534-805c-2e52dc4cf345'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:28.019100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0788d395b4e4632b29e263608e28973', 'request_id': 'req-5362f28c-ff89-4534-805c-2e52dc4cf345'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:28.035564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] The token cache is being invalidate because project a0788d395b4e4632b29e263608e28973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:28.036086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:28.037221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5362f28c-ff89-4534-805c-2e52dc4cf345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:28.039910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1049/2103] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:27 2022] DELETE /identity/v3/projects/a0788d395b4e4632b29e263608e28973 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:28.046339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c0ff-9829-45db-b019-afd751abfd15 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.047482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c0ff-9829-45db-b019-afd751abfd15 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.047482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c0ff-9829-45db-b019-afd751abfd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.083715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-8445c0ff-9829-45db-b019-afd751abfd15 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:28.085813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8445c0ff-9829-45db-b019-afd751abfd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:28.180446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1055/2104] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:28.188988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34798623-dd4a-40b5-9ec2-394103f9f855 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.189575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34798623-dd4a-40b5-9ec2-394103f9f855 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.190452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34798623-dd4a-40b5-9ec2-394103f9f855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.222826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-34798623-dd4a-40b5-9ec2-394103f9f855 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:28.222826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-34798623-dd4a-40b5-9ec2-394103f9f855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:28.236510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1050/2105] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:28.237769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445c0ff-9829-45db-b019-afd751abfd15 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:28.259190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445c0ff-9829-45db-b019-afd751abfd15 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:28.268796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:28.269667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.270112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.270325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.271093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:28.271475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:28.275865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:28.300821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34798623-dd4a-40b5-9ec2-394103f9f855 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:28.301797 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1056/2106] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:28.323669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34798623-dd4a-40b5-9ec2-394103f9f855 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:28.331692 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:28.333465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.334147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.334682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.336897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:28.345250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:28.388587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1051/2107] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:28 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:28.392723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:28.405307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed426-b7dc-4691-9bf6-13dc48a1fac1 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:28.561422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:28.574954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:28.575981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.576548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.576864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.581585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:28.584708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:28.665808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1057/2108] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:28 2022] POST /identity/v3/users => generated 289 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:28.669958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:28.695144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca54d7d-a468-4fe8-84b0-6db13178f7cb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:28.709803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:28.711332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.711655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.712123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.713285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:28.715239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:28.728517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1052/2109] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:28.732717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:28.743435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76f2b1-9ef0-4915-a586-7014b2b45653 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:28.749314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a8d4a-bebf-4c88-9871-b7d985e22aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:28.750004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a8d4a-bebf-4c88-9871-b7d985e22aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.750283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a8d4a-bebf-4c88-9871-b7d985e22aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.750561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a8d4a-bebf-4c88-9871-b7d985e22aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.751333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a8d4a-bebf-4c88-9871-b7d985e22aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:28.752987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a8d4a-bebf-4c88-9871-b7d985e22aa3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:28.767826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1058/2110] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:28.772053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:28.784522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d52d0-5dbf-471d-b783-31af447e9dfa admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:28.789305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:28.790213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.790498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.790774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] PATH_INFO: `/v3/projects/3c94142f5fe543e28710c9bb2916662c/users/b424792631ac40f48c3f3c1529d699b3/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.808126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c94142f5fe543e28710c9bb2916662c, user_id=b424792631ac40f48c3f3c1529d699b3, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:28.810912 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:28.827815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1053/2111] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:28 2022] PUT /identity/v3/projects/3c94142f5fe543e28710c9bb2916662c/users/b424792631ac40f48c3f3c1529d699b3/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:28.831437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c4bf4-e056-4205-8d28-4db62e60f86d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:28.831735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c4bf4-e056-4205-8d28-4db62e60f86d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:28.832022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c4bf4-e056-4205-8d28-4db62e60f86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:28.852369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-208c4bf4-e056-4205-8d28-4db62e60f86d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:28.852792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-208c4bf4-e056-4205-8d28-4db62e60f86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:28.924480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1059/2112] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:28.931311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:28.933365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:29.010621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558dc1d-3091-4b16-84a1-34b9a9df2eab admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:29.019354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:29.019947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:29.020141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:29.020394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:29.036193 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:29.037756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:29.053201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1054/2113] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:32.253738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecabb85-c525-407b-9602-970b68753c6d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:32.254760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecabb85-c525-407b-9602-970b68753c6d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:32.255986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecabb85-c525-407b-9602-970b68753c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:32.317139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0ecabb85-c525-407b-9602-970b68753c6d None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:32.320202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0ecabb85-c525-407b-9602-970b68753c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:32.439152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1060/2114] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:32.514091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:32.516469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:32.579608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:32.655190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13d59-f6f5-440f-aa12-84740abaefdf service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:32.663003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:32.664299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:32.664657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:32.665204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:32.676369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:32.678757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:32.691677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1055/2115] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 14:59:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 14:59:33.552033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c2c9-a150-4529-86cb-8ec0ae236a2a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:33.552567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c2c9-a150-4529-86cb-8ec0ae236a2a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:33.552567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51c2c9-a150-4529-86cb-8ec0ae236a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:33.572954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-4a51c2c9-a150-4529-86cb-8ec0ae236a2a None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:33.574396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4a51c2c9-a150-4529-86cb-8ec0ae236a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:33.582007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1061/2116] 10.222.0.72 () {58 vars in 967 bytes} [Thu Jun 16 14:59:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:33.596273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:33.613791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59faf0-4d04-4874-a07c-c713f26267f9 service swift] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:33.619541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b39ad2-77b1-46f7-899f-c12dc056d2da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:33.620941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b39ad2-77b1-46f7-899f-c12dc056d2da service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:33.620941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b39ad2-77b1-46f7-899f-c12dc056d2da service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:33.621244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b39ad2-77b1-46f7-899f-c12dc056d2da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:33.631546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b39ad2-77b1-46f7-899f-c12dc056d2da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:33.633587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b39ad2-77b1-46f7-899f-c12dc056d2da service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:33.650175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1056/2117] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 14:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 14:59:33.996669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:34.000057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:34.000625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:34.059248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] MFA Rules not processed for user `b424792631ac40f48c3f3c1529d699b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:34.060967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:34.074254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd37a-801b-4e76-81b4-a135bebcff3e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:34.074572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd37a-801b-4e76-81b4-a135bebcff3e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:34.074851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd37a-801b-4e76-81b4-a135bebcff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:34.097075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-934bd37a-801b-4e76-81b4-a135bebcff3e None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:34.098625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-934bd37a-801b-4e76-81b4-a135bebcff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:34.107311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:34.117290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1057/2118] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:34.138670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd37a-801b-4e76-81b4-a135bebcff3e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:34.154582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd37a-801b-4e76-81b4-a135bebcff3e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:34.161067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c41daf-1826-401f-bcca-0faee49b068b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:34.161989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c41daf-1826-401f-bcca-0faee49b068b service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:34.162440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c41daf-1826-401f-bcca-0faee49b068b service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:34.162873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c41daf-1826-401f-bcca-0faee49b068b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:34.173420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c41daf-1826-401f-bcca-0faee49b068b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:34.176516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c41daf-1826-401f-bcca-0faee49b068b service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:34.181584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1062/2119] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:34.187657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1058/2120] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 14:59:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 14:59:34.204751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:34.218251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:34.432591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629cd57-56ed-44ac-8aab-9a04882a8fd5 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:34.443095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:34.444299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:34.444924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:34.445333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:34.467666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:34.469683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:34.508644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1063/2121] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:34.729526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064a642-d77e-4d1b-b39c-25803ffd0180 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:34.730026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064a642-d77e-4d1b-b39c-25803ffd0180 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:34.730187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064a642-d77e-4d1b-b39c-25803ffd0180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:34.755021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c064a642-d77e-4d1b-b39c-25803ffd0180 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:34.756530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c064a642-d77e-4d1b-b39c-25803ffd0180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:34.772399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1059/2122] 10.222.0.72 () {58 vars in 967 bytes} [Thu Jun 16 14:59:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:34.797170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:34.815010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657c6ec-6f6b-4e92-a1a0-6f5d90a24560 service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:34.841933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:34.842620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:34.843005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:34.843283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:34.860933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:34.863031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:34.884197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1064/2123] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 14:59:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 14:59:36.208304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:36.208907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:36.208907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:36.246441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] MFA Rules not processed for user `b424792631ac40f48c3f3c1529d699b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:36.247968 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:36.273853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1060/2124] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:40.538537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:40.541095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:40.547392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:40.548909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:40.569805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77f46-3f91-499c-9520-2175930fa479 service swift] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:40.571609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb49abb-e77a-4eef-8ae3-4737d423868c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:40.575871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:40.577335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:40.577774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:40.577916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:40.578569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] PATH_INFO: `/v3/users/04f698df0a07477e833701ea15139321` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:40.580227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04f698df0a07477e833701ea15139321)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:40.582179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:40.602031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:40.602317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:40.608364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] PATH_INFO: `/v3/users/0030bdbe0d0f466a97b37f4f5a83b02f` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:40.610104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0030bdbe0d0f466a97b37f4f5a83b02f)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:40.615457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:40.686261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04f698df0a07477e833701ea15139321'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.687295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04f698df0a07477e833701ea15139321'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.688583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04f698df0a07477e833701ea15139321'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.716666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04f698df0a07477e833701ea15139321'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.718764 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0030bdbe0d0f466a97b37f4f5a83b02f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.720085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0030bdbe0d0f466a97b37f4f5a83b02f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.734379 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:40.737634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4b141d73-9b9e-4d7e-a620-884d67996934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04f698df0a07477e833701ea15139321'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.743390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0030bdbe0d0f466a97b37f4f5a83b02f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.745044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0030bdbe0d0f466a97b37f4f5a83b02f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.763471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5f3936ad-bf18-46aa-bd6c-67340d723d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0030bdbe0d0f466a97b37f4f5a83b02f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:40.769961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1065/2125] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:40 2022] DELETE /identity/v3/users/04f698df0a07477e833701ea15139321 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:40.774002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:40.774451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:40.774622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:40.798330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:40.799605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:40.830055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1061/2126] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 14:59:40 2022] DELETE /identity/v3/users/0030bdbe0d0f466a97b37f4f5a83b02f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:40.835329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:40.835667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:40.835904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:40.862541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:40.864405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:40.895540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1066/2127] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:40.905962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:40.907197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:40.907498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:40.978380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1062/2128] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:40.991678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:40.993989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:41.019733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b4963-a9f5-4464-84df-a50409f6a5ce None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:41.082903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a134c6f-3a2d-48eb-9ad1-bde121373baf None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:41.088612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:41.088907 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfcf08-8ecf-4b12-8cbf-d335e3cf9c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:41.089399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.089687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfcf08-8ecf-4b12-8cbf-d335e3cf9c74 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.089842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.090094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfcf08-8ecf-4b12-8cbf-d335e3cf9c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.090247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.090358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfcf08-8ecf-4b12-8cbf-d335e3cf9c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.102477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:41.103734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfcf08-8ecf-4b12-8cbf-d335e3cf9c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:41.103988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:41.105231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfcf08-8ecf-4b12-8cbf-d335e3cf9c74 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:41.116271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1067/2129] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:59:41.118847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1063/2130] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:59:41.490496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:41.492637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:41.520813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793812d-b126-4268-a7dd-8a3200709cca service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:41.541533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:41.543113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.543496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.543886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] PATH_INFO: `/v3/projects/22a75da564c3434ca8a1cff32b9edd4d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.545302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a75da564c3434ca8a1cff32b9edd4d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:41.547362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:41.604718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:41.611667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a75da564c3434ca8a1cff32b9edd4d', 'request_id': 'req-34f6bfed-f25c-4283-9bc9-a32d56f20570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:41.620737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a75da564c3434ca8a1cff32b9edd4d', 'request_id': 'req-34f6bfed-f25c-4283-9bc9-a32d56f20570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:41.629173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a75da564c3434ca8a1cff32b9edd4d', 'request_id': 'req-34f6bfed-f25c-4283-9bc9-a32d56f20570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:41.642457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] The token cache is being invalidate because project 22a75da564c3434ca8a1cff32b9edd4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:41.642752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:41.643524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-34f6bfed-f25c-4283-9bc9-a32d56f20570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:41.648237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1068/2131] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:41 2022] DELETE /identity/v3/projects/22a75da564c3434ca8a1cff32b9edd4d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:41.650901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb5db-a5b9-4ef6-a753-b082ea4a72e8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.651861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb5db-a5b9-4ef6-a753-b082ea4a72e8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.652294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb5db-a5b9-4ef6-a753-b082ea4a72e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.676664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-11fcb5db-a5b9-4ef6-a753-b082ea4a72e8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:41.678222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-11fcb5db-a5b9-4ef6-a753-b082ea4a72e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:41.766130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1064/2132] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:41.775841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.776123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.776398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.820823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:41.822594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:41.863758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1069/2133] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:41.866883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcb5db-a5b9-4ef6-a753-b082ea4a72e8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:41.874100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:41.877914 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:41.881304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcb5db-a5b9-4ef6-a753-b082ea4a72e8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:41.892367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c41d6c-5bdf-4b51-8a4e-4d042b190021 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:41.900601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:41.901592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.902072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.903331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.904275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:41.904754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:41.906636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:41.916396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:41.917648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.918104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.918740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] PATH_INFO: `/v3/projects/51bee9877ae34a1e95bf97e718159b2c` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.921119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] RBAC: Authorizing `identity:delete_project(project_id=51bee9877ae34a1e95bf97e718159b2c)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:41.924345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:41.959181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1065/2134] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:41.960705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:41.979735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4fc42-5a92-4fcc-a30f-4a86cc267292 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:41.988760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:41.989648 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:41.989942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:41.990223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:41.991492 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:41.993533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.019827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51bee9877ae34a1e95bf97e718159b2c', 'request_id': 'req-1eea8eaa-4945-4019-ae75-5a0082ee935b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:42.029499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1066/2135] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:41 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:42.029711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51bee9877ae34a1e95bf97e718159b2c', 'request_id': 'req-1eea8eaa-4945-4019-ae75-5a0082ee935b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:42.033825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.046792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51bee9877ae34a1e95bf97e718159b2c', 'request_id': 'req-1eea8eaa-4945-4019-ae75-5a0082ee935b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:42.061096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] The token cache is being invalidate because project 51bee9877ae34a1e95bf97e718159b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 14:59:42.061471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:42.062643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1eea8eaa-4945-4019-ae75-5a0082ee935b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 14:59:42.068588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1070/2136] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 14:59:41 2022] DELETE /identity/v3/projects/51bee9877ae34a1e95bf97e718159b2c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:42.071746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb20d3e-76ae-41cb-9d13-0297bc784e99 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.072168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb20d3e-76ae-41cb-9d13-0297bc784e99 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.072484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb20d3e-76ae-41cb-9d13-0297bc784e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.093504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5cb20d3e-76ae-41cb-9d13-0297bc784e99 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:42.094739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5cb20d3e-76ae-41cb-9d13-0297bc784e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:42.101340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:42.122893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff6710-43e1-40db-bc2b-c05be082d18a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.128058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.128712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.129032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.129326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.130033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.132735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.158168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1067/2137] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:42 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:42.162604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.165384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:42.179650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e33eb-f932-464a-8965-e62707f5b343 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.192921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.193113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1071/2138] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:42.193614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.193900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.194243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.195226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.196881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.205312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db756-2019-4ae0-bfaf-33534144fc0c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.205628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db756-2019-4ae0-bfaf-33534144fc0c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.205907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9db756-2019-4ae0-bfaf-33534144fc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.216817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1068/2139] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:42.218665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.226119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5d9db756-2019-4ae0-bfaf-33534144fc0c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:42.227355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5d9db756-2019-4ae0-bfaf-33534144fc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:42.231394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88ba3d-fad3-40f9-9f98-abfa26adf759 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.235621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1072/2140] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:42.236592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.237574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.237864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.240244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.240739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.242403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9db756-2019-4ae0-bfaf-33534144fc0c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.253015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1069/2141] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:42.256290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9db756-2019-4ae0-bfaf-33534144fc0c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.257418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.261570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.262265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.262587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.263063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.263574 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 14:59:42.264018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.265579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.268054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354374b-6702-487b-aac7-ae911bfac125 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.274005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a0bb7-9d62-4af4-8338-8159cda869cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.274917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0bb7-9d62-4af4-8338-8159cda869cb admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.275205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0bb7-9d62-4af4-8338-8159cda869cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.275495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a0bb7-9d62-4af4-8338-8159cda869cb admin admin] PATH_INFO: `/v3/projects/05223500b4a142c7a77c2de010541b84/users/9d977f609537476daa8fe8adbab9ff9c/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.278776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1073/2142] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 14:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 14:59:42.283279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.292293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a0bb7-9d62-4af4-8338-8159cda869cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=05223500b4a142c7a77c2de010541b84, user_id=9d977f609537476daa8fe8adbab9ff9c, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.293844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e392be-00b1-4e1f-ae5a-c42a8e1915e3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.295066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a0bb7-9d62-4af4-8338-8159cda869cb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.298637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.299299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.299635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.299956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.300964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.304147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.317245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1070/2143] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:42 2022] PUT /identity/v3/projects/05223500b4a142c7a77c2de010541b84/users/9d977f609537476daa8fe8adbab9ff9c/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:42.324599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.325161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.325658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.339977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1074/2144] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:42 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 14:59:42.341306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.347187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:42.348418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:42.351343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:42.421368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1071/2145] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 14:59:42.428298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a11a32-67ef-4449-a498-ab7824087d7e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.435264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.436911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:42.440777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.441874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.442327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.442762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.443934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.449906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.492913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1075/2146] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:42 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:42.496071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.508761 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3d27-1e5d-4b9e-90df-92b196c57431 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.517393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.518137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.518419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.518693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.519434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.521200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.540557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1076/2147] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:42.540884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ef257-2c7e-4e8c-bd4b-4c2f76108a3b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.541927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.549099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.549859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.550283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.550995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.552569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfafeb-9b5a-49cc-b0d4-6b9f9367a228 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.561142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024b9a8-4365-4d29-9f8a-f47fb3774a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.562157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024b9a8-4365-4d29-9f8a-f47fb3774a32 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.562570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024b9a8-4365-4d29-9f8a-f47fb3774a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.563029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024b9a8-4365-4d29-9f8a-f47fb3774a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.564052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024b9a8-4365-4d29-9f8a-f47fb3774a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.564476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.565960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.566170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024b9a8-4365-4d29-9f8a-f47fb3774a32 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.579175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1072/2148] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:42.587257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1077/2149] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:42.591023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.610135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59317423-e9fd-44b2-9de8-ec8ad5a569e0 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.615850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.616769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.617076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.617347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] PATH_INFO: `/v3/projects/8bfa0f9f3251447c922938888ddea917/users/aab9d86304804cb189c0c1d065d7e7b1/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.637938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bfa0f9f3251447c922938888ddea917, user_id=aab9d86304804cb189c0c1d065d7e7b1, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.641095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.654206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:42.663756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1073/2150] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:42 2022] PUT /identity/v3/projects/8bfa0f9f3251447c922938888ddea917/users/aab9d86304804cb189c0c1d065d7e7b1/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:42.671573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bc616-4ba2-45ac-bc3e-9847337b3d2c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.672182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bc616-4ba2-45ac-bc3e-9847337b3d2c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.672957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bc616-4ba2-45ac-bc3e-9847337b3d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.696678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-af3bc616-4ba2-45ac-bc3e-9847337b3d2c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:42.697963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-af3bc616-4ba2-45ac-bc3e-9847337b3d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:42.776448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1078/2151] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:42.787393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:42.790283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:42.884093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6387c0-0eee-474d-8930-c4bb0abf942e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:42.888530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:42.889239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:42.889517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:42.889790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:42.902779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:42.904629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:42.927158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1074/2152] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 14:59:43.812428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15de66-cb77-4f4e-8977-fe95b081b2d7 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:43.813737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15de66-cb77-4f4e-8977-fe95b081b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:43.814283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15de66-cb77-4f4e-8977-fe95b081b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:43.861643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-cc15de66-cb77-4f4e-8977-fe95b081b2d7 None None] MFA Rules not processed for user `aab9d86304804cb189c0c1d065d7e7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:43.863205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cc15de66-cb77-4f4e-8977-fe95b081b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:43.959409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1079/2153] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:43.974859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:43.977791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:44.061740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529faff-8c7c-4033-9431-a8929c87e2a6 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:44.073657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:44.076670 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:44.077546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:44.078911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:44.106097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:44.114792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:44.145367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1075/2154] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 14:59:48.206038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb0db-c86f-4108-89c5-dd2b131f0074 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:48.206567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb0db-c86f-4108-89c5-dd2b131f0074 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:48.206896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb0db-c86f-4108-89c5-dd2b131f0074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:48.215438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:48.244502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-0f0eb0db-c86f-4108-89c5-dd2b131f0074 None None] MFA Rules not processed for user `9d977f609537476daa8fe8adbab9ff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:48.245957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0f0eb0db-c86f-4108-89c5-dd2b131f0074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:48.348975 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1080/2155] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:48.354205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:48.407576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342ab0d-0ae5-40da-8df3-04f6be49dc93 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:48.421586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:48.422272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:48.422552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:48.422823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:48.423535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:48.429001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:48.479543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1076/2156] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 14:59:48 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:48.484224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb0db-c86f-4108-89c5-dd2b131f0074 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:48.509363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb0db-c86f-4108-89c5-dd2b131f0074 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:48.611827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:48.613474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:48.613474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:48.613474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:48.614192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:48.616310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:48.663800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1081/2157] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 14:59:48 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 14:59:48.668039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:48.682885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa630-7251-441c-b1d0-1217ad88b914 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:48.688587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:48.689514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:48.689896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:48.690245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:48.691188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:48.694830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:48.708981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1077/2158] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 14:59:48.714387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:48.727281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7fd53-555d-4c8f-b309-beff99c9ba75 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:48.735072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:48.736059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:48.736344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:48.736632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] PATH_INFO: `/v3/projects/0531c85fd17e44fcbac42ff8e5496707/users/4b3012ed20354085ac032f78a2fa91b8/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:48.755625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0531c85fd17e44fcbac42ff8e5496707, user_id=4b3012ed20354085ac032f78a2fa91b8, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:48.759218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:48.778927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1082/2159] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:48 2022] PUT /identity/v3/projects/0531c85fd17e44fcbac42ff8e5496707/users/4b3012ed20354085ac032f78a2fa91b8/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:48.785458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:48.824236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebcda5-ff94-421e-9046-2d0e7f96c506 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:48.923476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7ee41-d7fe-42a2-9e7d-1aa8fede8b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:48.924291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7ee41-d7fe-42a2-9e7d-1aa8fede8b71 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:48.924579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7ee41-d7fe-42a2-9e7d-1aa8fede8b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:48.924972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7ee41-d7fe-42a2-9e7d-1aa8fede8b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:48.925877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7ee41-d7fe-42a2-9e7d-1aa8fede8b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:48.927771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7ee41-d7fe-42a2-9e7d-1aa8fede8b71 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:48.943547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1078/2160] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 14:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 14:59:48.949350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:48.962876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaaafc-b43b-4a27-ba16-8d5616d9b104 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:48.969030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3acb791-4439-46c8-99b3-93b5775eec21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:48.969874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb791-4439-46c8-99b3-93b5775eec21 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:48.970160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb791-4439-46c8-99b3-93b5775eec21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:48.970504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3acb791-4439-46c8-99b3-93b5775eec21 admin admin] PATH_INFO: `/v3/projects/0531c85fd17e44fcbac42ff8e5496707/users/4b3012ed20354085ac032f78a2fa91b8/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:48.972600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb791-4439-46c8-99b3-93b5775eec21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0531c85fd17e44fcbac42ff8e5496707, user_id=4b3012ed20354085ac032f78a2fa91b8, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:48.975494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3acb791-4439-46c8-99b3-93b5775eec21 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:48.997501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1083/2161] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 14:59:48 2022] PUT /identity/v3/projects/0531c85fd17e44fcbac42ff8e5496707/users/4b3012ed20354085ac032f78a2fa91b8/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 14:59:49.132037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:49.132393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:49.132920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:49.163703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] MFA Rules not processed for user `aab9d86304804cb189c0c1d065d7e7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:49.165003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:49.173366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:49.278740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1079/2162] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:55.224622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:55.225750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:55.225884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:55.258055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] MFA Rules not processed for user `4b3012ed20354085ac032f78a2fa91b8`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 14:59:55.259295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:55.329642 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 14:59:55.417267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1084/2163] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 14:59:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 14:59:55.434496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 14:59:55.436538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 14:59:55.609407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f950cd-af23-4585-b9f3-fbe43ed8382a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 14:59:55.617284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac7f12-59b8-4658-94f8-6f4c1124419c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 14:59:55.618050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac7f12-59b8-4658-94f8-6f4c1124419c service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 14:59:55.618410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac7f12-59b8-4658-94f8-6f4c1124419c service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 14:59:55.618761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac7f12-59b8-4658-94f8-6f4c1124419c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 14:59:55.727659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac7f12-59b8-4658-94f8-6f4c1124419c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 14:59:55.729707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac7f12-59b8-4658-94f8-6f4c1124419c service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 14:59:55.763958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1080/2164] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:00:00.934194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:00.937918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:01.059395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff5d92-3dc0-4ee2-a5fb-d1e252ebea0e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:01.070213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:01.071429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:01.071884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:01.072328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] PATH_INFO: `/v3/users/5b662d94c0f54f26b8283cdc22770df7` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:01.074087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b662d94c0f54f26b8283cdc22770df7)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:01.077961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:01.190958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b662d94c0f54f26b8283cdc22770df7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:01.192579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b662d94c0f54f26b8283cdc22770df7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:01.195694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b662d94c0f54f26b8283cdc22770df7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:01.226629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b662d94c0f54f26b8283cdc22770df7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:01.249014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b662d94c0f54f26b8283cdc22770df7'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:01.292070 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1085/2165] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:00:00 2022] DELETE /identity/v3/users/5b662d94c0f54f26b8283cdc22770df7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:00:01.296439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:01.297276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:01.297276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:01.327766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:01.330095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:01.405997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:01.430495 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1081/2166] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:01.447398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:01.449663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:01.544619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a648b-2886-44c2-b642-bf9dabfaa40e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:01.551274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:01.552365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:01.552902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:01.553388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:01.568595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:01.573846 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:01.576082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:01.594531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1086/2167] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:00:01.979093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:01.979918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:02.001788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd777ac4-d0cb-410a-81fe-5bdf26fc227a None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:02.015688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:02.016684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:02.017290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:02.018466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] PATH_INFO: `/v3/projects/b475cd8351374fb0a0853b4d4534dfd5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:02.026102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b475cd8351374fb0a0853b4d4534dfd5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:02.026102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:02.113459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b475cd8351374fb0a0853b4d4534dfd5', 'request_id': 'req-315da8d4-4c95-44a5-9b4c-3ecb49de8529'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:02.119445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b475cd8351374fb0a0853b4d4534dfd5', 'request_id': 'req-315da8d4-4c95-44a5-9b4c-3ecb49de8529'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:02.130886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b475cd8351374fb0a0853b4d4534dfd5', 'request_id': 'req-315da8d4-4c95-44a5-9b4c-3ecb49de8529'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:02.153650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] The token cache is being invalidate because project b475cd8351374fb0a0853b4d4534dfd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:00:02.153987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:02.154733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-315da8d4-4c95-44a5-9b4c-3ecb49de8529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:02.156500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1082/2168] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:00:01 2022] DELETE /identity/v3/projects/b475cd8351374fb0a0853b4d4534dfd5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:00:11.590423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:12.144864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:23.673459 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:23.675959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:23.794472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f02f6-cb04-4fac-80a1-d11d596ed326 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:23.868712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ff9dc-08b0-4abc-8bf8-e4d59c5a5d75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:23.871131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ff9dc-08b0-4abc-8bf8-e4d59c5a5d75 service placement] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:23.871540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ff9dc-08b0-4abc-8bf8-e4d59c5a5d75 service placement] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:23.871918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ff9dc-08b0-4abc-8bf8-e4d59c5a5d75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:23.874390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cb6ff9dc-08b0-4abc-8bf8-e4d59c5a5d75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:23.902723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ff9dc-08b0-4abc-8bf8-e4d59c5a5d75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:23.904205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ff9dc-08b0-4abc-8bf8-e4d59c5a5d75 service placement] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:23.916331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1087/2169] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:00:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:00:33.910938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=58 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:39.631177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:39.631688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:39.631802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:39.650002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] MFA Rules not processed for user `aab9d86304804cb189c0c1d065d7e7b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:39.651236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:39.706671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1083/2170] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:41.516209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64712005-0729-4fe9-9aff-7ed516178451 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:41.517315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64712005-0729-4fe9-9aff-7ed516178451 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:41.517315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64712005-0729-4fe9-9aff-7ed516178451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:41.536303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-64712005-0729-4fe9-9aff-7ed516178451 None None] MFA Rules not processed for user `9d977f609537476daa8fe8adbab9ff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:41.537521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-64712005-0729-4fe9-9aff-7ed516178451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:41.618509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1088/2171] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:41.754353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:41.755783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:41.841354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d693c-90f2-41b1-98d4-974a53587864 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:41.846139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:41.847035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:41.847319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:41.847593 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] PATH_INFO: `/v3/users/fafc3468763148f4a4fb3b3216156e97` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:41.848794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafc3468763148f4a4fb3b3216156e97)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:41.850763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:41.921717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafc3468763148f4a4fb3b3216156e97'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:41.922159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafc3468763148f4a4fb3b3216156e97'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:41.930953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafc3468763148f4a4fb3b3216156e97'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:41.931708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafc3468763148f4a4fb3b3216156e97'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:41.941378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafc3468763148f4a4fb3b3216156e97'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:41.963689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1084/2172] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:00:41 2022] DELETE /identity/v3/users/fafc3468763148f4a4fb3b3216156e97 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:00:41.973428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb18748-29af-44ce-9124-130e57095dff None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:41.973650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb18748-29af-44ce-9124-130e57095dff None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:41.974166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb18748-29af-44ce-9124-130e57095dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:41.996551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-dfb18748-29af-44ce-9124-130e57095dff None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:41.998118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dfb18748-29af-44ce-9124-130e57095dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:42.069254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1089/2173] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:42.082650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:42.084759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:42.119884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:42.157793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149dc9c-3535-42a8-a02b-7d7bb2b202da admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:42.163090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724bd64-944a-400b-980c-ae19ffb799e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:42.163791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724bd64-944a-400b-980c-ae19ffb799e9 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:42.164086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724bd64-944a-400b-980c-ae19ffb799e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:42.164917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724bd64-944a-400b-980c-ae19ffb799e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:42.175291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724bd64-944a-400b-980c-ae19ffb799e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:42.176748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724bd64-944a-400b-980c-ae19ffb799e9 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:42.188212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1085/2174] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:00:42.936943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb18748-29af-44ce-9124-130e57095dff None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:42.938919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dfb18748-29af-44ce-9124-130e57095dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:42.952072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb18748-29af-44ce-9124-130e57095dff None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:42.958346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:42.959363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:42.959586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:42.959861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] PATH_INFO: `/v3/projects/e15c04f0160548c4b1927d4f04464fb9` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:42.961184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e15c04f0160548c4b1927d4f04464fb9)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:42.963324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.033507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e15c04f0160548c4b1927d4f04464fb9', 'request_id': 'req-022e9b1d-6d13-447d-9237-09d42d3ef11b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:43.044727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e15c04f0160548c4b1927d4f04464fb9', 'request_id': 'req-022e9b1d-6d13-447d-9237-09d42d3ef11b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:43.056442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e15c04f0160548c4b1927d4f04464fb9', 'request_id': 'req-022e9b1d-6d13-447d-9237-09d42d3ef11b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:43.073504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] The token cache is being invalidate because project e15c04f0160548c4b1927d4f04464fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:00:43.073987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:43.075092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-022e9b1d-6d13-447d-9237-09d42d3ef11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:43.077632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1090/2175] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:00:42 2022] DELETE /identity/v3/projects/e15c04f0160548c4b1927d4f04464fb9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:00:43.092424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac695c-f9c6-42cc-9749-b511cab40eb4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.093576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac695c-f9c6-42cc-9749-b511cab40eb4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.094217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac695c-f9c6-42cc-9749-b511cab40eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.113542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-71ac695c-f9c6-42cc-9749-b511cab40eb4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:43.114872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-71ac695c-f9c6-42cc-9749-b511cab40eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:43.183000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1086/2176] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:43.190525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4244-bddd-4031-864c-2ced21be6a71 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.190959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4244-bddd-4031-864c-2ced21be6a71 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.191446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a4244-bddd-4031-864c-2ced21be6a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.212437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-379a4244-bddd-4031-864c-2ced21be6a71 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:43.213619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-379a4244-bddd-4031-864c-2ced21be6a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:43.222130 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1091/2177] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:43.226402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac695c-f9c6-42cc-9749-b511cab40eb4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:43.238255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac695c-f9c6-42cc-9749-b511cab40eb4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:43.243249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:43.244260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.244462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.244933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.245871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:00:43.246360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:43.248578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.261961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1087/2178] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:00:43.266331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a4244-bddd-4031-864c-2ced21be6a71 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:43.276457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a4244-bddd-4031-864c-2ced21be6a71 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:43.282507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:43.283179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.283466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.283746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.284446 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:43.286461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.308790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:43.318692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1092/2179] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:00:43 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:00:43.323117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:43.334200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764d047-9daf-4b61-9a7b-df2faf274204 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:43.401922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:43.402153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.402291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.402559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.403251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:43.405441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.425323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1088/2180] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:00:43 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:00:43.429412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:43.439813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6f7b4-0505-4f17-93e9-804b52369192 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:43.444397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:43.445112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.445402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.445680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.446406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:43.448010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.458522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1093/2181] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:00:43.463131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:43.473232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b5f8a-44fb-44d5-b6a9-5c35a3b07fe7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:43.478542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f97760-3b85-42ef-b1db-e2a13385e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:43.479194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f97760-3b85-42ef-b1db-e2a13385e47c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.479511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f97760-3b85-42ef-b1db-e2a13385e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.479876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f97760-3b85-42ef-b1db-e2a13385e47c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.480562 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f97760-3b85-42ef-b1db-e2a13385e47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:43.482233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f97760-3b85-42ef-b1db-e2a13385e47c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.493110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1089/2182] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:00:43.497210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:43.507189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1df2fe-d5ec-40f1-8e60-3cdf16d50c8e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:43.511821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:43.512705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.513033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.513312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] PATH_INFO: `/v3/projects/2325339e8ac54a6ba53ec27c02aaf854/users/a878c01620d34a30b721bb9bade34991/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.531341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2325339e8ac54a6ba53ec27c02aaf854, user_id=a878c01620d34a30b721bb9bade34991, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:43.534581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.549381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1094/2183] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:00:43 2022] PUT /identity/v3/projects/2325339e8ac54a6ba53ec27c02aaf854/users/a878c01620d34a30b721bb9bade34991/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:00:43.560126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d8662-75e1-4bd0-b976-e624c0280b40 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.560568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d8662-75e1-4bd0-b976-e624c0280b40 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.561006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d8662-75e1-4bd0-b976-e624c0280b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.589894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c26d8662-75e1-4bd0-b976-e624c0280b40 None None] MFA Rules not processed for user `a878c01620d34a30b721bb9bade34991`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:43.591152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c26d8662-75e1-4bd0-b976-e624c0280b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:43.614305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:43.667309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1090/2184] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:00:43.678075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:43.679902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:43.752649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80350992-4470-4922-b6a6-c5703195a3e0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:43.759655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80263a9-2690-4d94-8738-974641292827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:43.761692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80263a9-2690-4d94-8738-974641292827 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:43.762085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80263a9-2690-4d94-8738-974641292827 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:43.762714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80263a9-2690-4d94-8738-974641292827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:43.775706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80263a9-2690-4d94-8738-974641292827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:43.777460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80263a9-2690-4d94-8738-974641292827 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:43.790209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1095/2185] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:00:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:00:53.663711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=17 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:53.785396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:55.154440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d8662-75e1-4bd0-b976-e624c0280b40 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:55.218678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d8662-75e1-4bd0-b976-e624c0280b40 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:55.223187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:55.224054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:55.224334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:55.224607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] PATH_INFO: `/v3/users/a878c01620d34a30b721bb9bade34991` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:55.225814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a878c01620d34a30b721bb9bade34991)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:55.227837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:55.274383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a878c01620d34a30b721bb9bade34991'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.275050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a878c01620d34a30b721bb9bade34991'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.282162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a878c01620d34a30b721bb9bade34991'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.282806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a878c01620d34a30b721bb9bade34991'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.290203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a878c01620d34a30b721bb9bade34991'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.307847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1091/2186] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:00:55 2022] DELETE /identity/v3/users/a878c01620d34a30b721bb9bade34991 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:00:55.312639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:55.313406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:55.313622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:55.331332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:55.332666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:55.393618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1096/2187] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:55.403263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:55.405251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:55.461953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e8e3d-1d74-498d-8e2d-e3c168cbae43 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:55.465712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cde7f-f86b-4edf-a9bc-67f740962a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:55.466359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cde7f-f86b-4edf-a9bc-67f740962a6c service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:55.466637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cde7f-f86b-4edf-a9bc-67f740962a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:55.466908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cde7f-f86b-4edf-a9bc-67f740962a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:55.476494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cde7f-f86b-4edf-a9bc-67f740962a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:55.478001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cde7f-f86b-4edf-a9bc-67f740962a6c service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:55.488536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1092/2188] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:00:55.839006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:55.840622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:55.853957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2b70e-1930-4f59-a184-5b0d085f314f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:55.859348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:55.860491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:55.860753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:55.861057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] PATH_INFO: `/v3/projects/2325339e8ac54a6ba53ec27c02aaf854` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:55.862397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2325339e8ac54a6ba53ec27c02aaf854)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:55.864325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:55.918850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2325339e8ac54a6ba53ec27c02aaf854', 'request_id': 'req-0856255e-4dc9-4e41-b458-06df0909c000'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.924634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2325339e8ac54a6ba53ec27c02aaf854', 'request_id': 'req-0856255e-4dc9-4e41-b458-06df0909c000'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.932240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2325339e8ac54a6ba53ec27c02aaf854', 'request_id': 'req-0856255e-4dc9-4e41-b458-06df0909c000'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.943352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] The token cache is being invalidate because project 2325339e8ac54a6ba53ec27c02aaf854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:00:55.943640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.944269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0856255e-4dc9-4e41-b458-06df0909c000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:00:55.945855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1097/2189] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:00:55 2022] DELETE /identity/v3/projects/2325339e8ac54a6ba53ec27c02aaf854 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:00:55.950899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66762e7-dc0b-4782-82a8-55561f3c9e6b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:55.951191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66762e7-dc0b-4782-82a8-55561f3c9e6b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:55.951577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66762e7-dc0b-4782-82a8-55561f3c9e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:55.970286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c66762e7-dc0b-4782-82a8-55561f3c9e6b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:55.971520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c66762e7-dc0b-4782-82a8-55561f3c9e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:55.988843 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:56.034258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1093/2190] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:56.043039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d428c-969d-4871-8c92-94e8a7a1ab5f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.043326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d428c-969d-4871-8c92-94e8a7a1ab5f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.043602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d428c-969d-4871-8c92-94e8a7a1ab5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.062354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-eb9d428c-969d-4871-8c92-94e8a7a1ab5f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:56.063580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-eb9d428c-969d-4871-8c92-94e8a7a1ab5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:56.073982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1098/2191] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:56.078177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66762e7-dc0b-4782-82a8-55561f3c9e6b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:56.088875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66762e7-dc0b-4782-82a8-55561f3c9e6b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:56.093526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:56.094188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.094469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.094747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.095476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:00:56.095792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:56.097496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:56.109823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1094/2192] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:00:56.114167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d428c-969d-4871-8c92-94e8a7a1ab5f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:56.123717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d428c-969d-4871-8c92-94e8a7a1ab5f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:56.128393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:56.129220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.129514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.129791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.130763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:56.132697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:56.158501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1099/2193] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:00:56 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:00:56.162565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:56.173499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36252764-577b-4e87-91a9-03fc3933ebdc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:56.238567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:56.239257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.239542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.239816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.240528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:56.242639 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:56.268282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1095/2194] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:00:56 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:00:56.272160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:56.284883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cce26a-47b6-464d-905d-bee9467e3ac1 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:56.290167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:56.290845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.291132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.291413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.292161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:56.293857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:56.309875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1100/2195] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:00:56.311150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:56.322321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397eae4-4c72-4030-84d1-093c5b43f95c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:56.327771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ced7e-3581-4a13-9c7b-93061fb5a80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:56.328453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ced7e-3581-4a13-9c7b-93061fb5a80b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.328737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ced7e-3581-4a13-9c7b-93061fb5a80b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.329048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ced7e-3581-4a13-9c7b-93061fb5a80b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.329785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ced7e-3581-4a13-9c7b-93061fb5a80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:56.331393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ced7e-3581-4a13-9c7b-93061fb5a80b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:56.343043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1096/2196] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:00:56.346934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:56.361079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1844-0547-4da0-b1f1-3d3c2c282f78 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:56.368971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:56.369859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.370136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.370403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] PATH_INFO: `/v3/projects/3048a9bbedf74d49a27c9911ca2ad960/users/d0b498bcfc434db99010657e2ab544b9/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.386235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3048a9bbedf74d49a27c9911ca2ad960, user_id=d0b498bcfc434db99010657e2ab544b9, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:56.388923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:56.402432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1101/2197] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:00:56 2022] PUT /identity/v3/projects/3048a9bbedf74d49a27c9911ca2ad960/users/d0b498bcfc434db99010657e2ab544b9/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:00:56.407757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6562797-d083-480d-b5e9-0fd278064083 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.408048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6562797-d083-480d-b5e9-0fd278064083 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.408330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6562797-d083-480d-b5e9-0fd278064083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.426102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b6562797-d083-480d-b5e9-0fd278064083 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:00:56.427225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b6562797-d083-480d-b5e9-0fd278064083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:56.486568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1097/2198] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:00:56.496404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:00:56.498181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:00:56.521251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:00:56.559083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b04d2-2940-493c-8a16-4a216f777daa admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:00:56.562704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:00:56.563348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:00:56.563628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:00:56.563899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:00:56.573880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:00:56.575561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:00:56.585017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1102/2199] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:01:01.330857 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6fc75-5d6c-497a-8536-d5fce4016f67 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:01.332378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6fc75-5d6c-497a-8536-d5fce4016f67 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:01.332881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6fc75-5d6c-497a-8536-d5fce4016f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:01.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-29c6fc75-5d6c-497a-8536-d5fce4016f67 None None] MFA Rules not processed for user `d0b498bcfc434db99010657e2ab544b9`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:01.384896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-29c6fc75-5d6c-497a-8536-d5fce4016f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:01.415221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:01.520160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1098/2200] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:01.524383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:01.537757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce97d-e363-4cd9-b9b3-038c5749496c service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:01.544791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:01.545600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:01.545941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:01.546299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:01.547051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:01.549287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:01.590386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6fc75-5d6c-497a-8536-d5fce4016f67 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:01.590695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1103/2201] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:01:01 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:01:01.603753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6fc75-5d6c-497a-8536-d5fce4016f67 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:01.687186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:01.688858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:01.689206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:01.689653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:01.690922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:01.694333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:01.737369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1099/2202] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:01:01 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:01:01.743856 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:01.768599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392dbf9-b11b-4058-99df-28727a0bca2d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:01.821869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:01.822769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:01.823400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:01.823834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:01.824747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:01.827939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:01.842286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1104/2203] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:01:01.847214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:01.870182 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb56c2-a017-4c1c-87af-3b8165f395b7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:01.875965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:01.877000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:01.877262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:01.877531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] PATH_INFO: `/v3/projects/c732e283521445d58a8d0445bc7d6759/users/35195eb3115f408ba9ab4c9a9d123570/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:01.909951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c732e283521445d58a8d0445bc7d6759, user_id=35195eb3115f408ba9ab4c9a9d123570, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:01.913157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:01.936006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1100/2204] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:01:01 2022] PUT /identity/v3/projects/c732e283521445d58a8d0445bc7d6759/users/35195eb3115f408ba9ab4c9a9d123570/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:01:01.937444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:01.949050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076641a-1af4-4367-86f2-5460281e602f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:02.024710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305319e-6e59-4d49-955e-1671833bcaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:02.025506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305319e-6e59-4d49-955e-1671833bcaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:02.025797 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305319e-6e59-4d49-955e-1671833bcaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:02.026109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305319e-6e59-4d49-955e-1671833bcaa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:02.026873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305319e-6e59-4d49-955e-1671833bcaa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:02.028550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305319e-6e59-4d49-955e-1671833bcaa5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:02.045600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1105/2205] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:01:02.046627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:02.058475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584c295-0f39-41f1-bdb0-b1e9a49972a6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:02.064516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:02.065698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:02.066049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:02.066421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] PATH_INFO: `/v3/projects/c732e283521445d58a8d0445bc7d6759/users/35195eb3115f408ba9ab4c9a9d123570/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:02.068872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c732e283521445d58a8d0445bc7d6759, user_id=35195eb3115f408ba9ab4c9a9d123570, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:02.078061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:02.093616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1101/2206] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:01:02 2022] PUT /identity/v3/projects/c732e283521445d58a8d0445bc7d6759/users/35195eb3115f408ba9ab4c9a9d123570/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:01:03.803221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1098d9-3f8c-4943-b885-2d970c91d3c2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:03.804231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1098d9-3f8c-4943-b885-2d970c91d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:03.804970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1098d9-3f8c-4943-b885-2d970c91d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:03.845281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5d1098d9-3f8c-4943-b885-2d970c91d3c2 None None] MFA Rules not processed for user `73a51a4e876e4910bbc5dd69f8a18058`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:03.846553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5d1098d9-3f8c-4943-b885-2d970c91d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:03.982197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1106/2207] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:08.343014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:08.344769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:08.427279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e2f01-4c32-4032-a2d4-4720edb4b240 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:08.432089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:08.432852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:08.433150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:08.433425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:08.435696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:08.488452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:08.565845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:08.567645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:08.604604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1102/2208] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:01:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:01:08.925742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaaf0a-a2d6-4299-a531-002af053830b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:08.926753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaaf0a-a2d6-4299-a531-002af053830b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:08.927102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaaf0a-a2d6-4299-a531-002af053830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:08.964516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b8aaaf0a-a2d6-4299-a531-002af053830b None None] MFA Rules not processed for user `35195eb3115f408ba9ab4c9a9d123570`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:08.966454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b8aaaf0a-a2d6-4299-a531-002af053830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:08.997641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:09.084814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1107/2209] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:09.098187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:09.100305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:09.236763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7846ee-5a55-44a7-9b42-e1e21c1be346 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:09.241012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:09.241761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:09.242156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:09.242820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:09.335128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:09.337509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:09.350168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1103/2210] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:01:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:01:09.485687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29320a6-b05b-40eb-992d-c2f192b96f69 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:09.488277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29320a6-b05b-40eb-992d-c2f192b96f69 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:09.490049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29320a6-b05b-40eb-992d-c2f192b96f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:09.524109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-e29320a6-b05b-40eb-992d-c2f192b96f69 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:09.525730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e29320a6-b05b-40eb-992d-c2f192b96f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:09.655965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1108/2211] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:01:09.714549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:09.716111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:09.791026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:09.810308 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e7234-4c16-42fa-bf1d-d786a15e1651 service glance] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:09.818778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:09.819455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:09.819736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:09.820033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:09.833323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:09.835044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:09.846363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1104/2212] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:01:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:01:12.717664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c580f-e784-4b7d-8a1e-1fbb9b3ba9c4 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:12.718610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c580f-e784-4b7d-8a1e-1fbb9b3ba9c4 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:12.719256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c580f-e784-4b7d-8a1e-1fbb9b3ba9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:12.749507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b73c580f-e784-4b7d-8a1e-1fbb9b3ba9c4 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:12.750757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b73c580f-e784-4b7d-8a1e-1fbb9b3ba9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:12.767814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1109/2213] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:12.791311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:12.807156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d69cf1-bb17-46ba-91c2-c9d5660b69ff service swift] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:12.814033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908adfd3-89be-4a87-877e-cd9e9edb8840 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:12.815080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908adfd3-89be-4a87-877e-cd9e9edb8840 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:12.816478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908adfd3-89be-4a87-877e-cd9e9edb8840 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:12.817482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908adfd3-89be-4a87-877e-cd9e9edb8840 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:12.832304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908adfd3-89be-4a87-877e-cd9e9edb8840 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:12.834270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908adfd3-89be-4a87-877e-cd9e9edb8840 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:12.845757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1105/2214] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:01:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:01:22.742258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=48 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:22.841624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=13 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:23.434479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a45f8c-03a7-4191-94fc-a271ba90efad None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:23.434971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a45f8c-03a7-4191-94fc-a271ba90efad None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:23.435472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a45f8c-03a7-4191-94fc-a271ba90efad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:23.507824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f5a45f8c-03a7-4191-94fc-a271ba90efad None None] MFA Rules not processed for user `d0b498bcfc434db99010657e2ab544b9`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:23.511151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f5a45f8c-03a7-4191-94fc-a271ba90efad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:23.528615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1110/2215] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:01:26.758917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7254-75a0-4fcf-a518-a67c8214fd68 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:26.759641 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7254-75a0-4fcf-a518-a67c8214fd68 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:26.759862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a7254-75a0-4fcf-a518-a67c8214fd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:26.791617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-e23a7254-75a0-4fcf-a518-a67c8214fd68 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:26.793373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e23a7254-75a0-4fcf-a518-a67c8214fd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:26.834470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1106/2216] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:26.848954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a45f8c-03a7-4191-94fc-a271ba90efad None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:26.897129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a45f8c-03a7-4191-94fc-a271ba90efad None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:26.931896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:26.933170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:26.937117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:26.937591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:26.978086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:26.981175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:27.027386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1111/2217] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:01:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 15:01:33.714801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:33.715419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:33.715419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:33.738008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:33.739291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:33.750621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1107/2218] 10.222.0.72 () {58 vars in 967 bytes} [Thu Jun 16 15:01:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:33.767518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:33.778188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69094eef-a938-4ffa-a8e3-54efc5ee8f91 service swift] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:33.784016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:33.784674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:33.785000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:33.785286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:33.794538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:33.796144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:33.808894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1112/2219] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:01:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:01:36.227774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:36.229737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:36.304092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcb5a0-06ec-4c00-acba-6d2aebe02bd9 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:36.309597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:36.310479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:36.310765 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:36.311036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] PATH_INFO: `/v3/users/aab9d86304804cb189c0c1d065d7e7b1` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:36.312245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab9d86304804cb189c0c1d065d7e7b1)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:36.314189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:36.376756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab9d86304804cb189c0c1d065d7e7b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.377629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab9d86304804cb189c0c1d065d7e7b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.387538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab9d86304804cb189c0c1d065d7e7b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.388675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab9d86304804cb189c0c1d065d7e7b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.398351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-48cd01e3-8afc-4b59-a325-9f5c3b172a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab9d86304804cb189c0c1d065d7e7b1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.418772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1108/2220] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:01:36 2022] DELETE /identity/v3/users/aab9d86304804cb189c0c1d065d7e7b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:01:36.614178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:36.632795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a0448-a212-4854-8eee-b0691e12aede service swift] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:36.747453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:36.748417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:36.748710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:36.749148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] PATH_INFO: `/v3/projects/8bfa0f9f3251447c922938888ddea917` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:36.750614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bfa0f9f3251447c922938888ddea917)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:36.753564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:36.820602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bfa0f9f3251447c922938888ddea917', 'request_id': 'req-f8292090-d26c-430d-84af-f406a05eb1b2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.827698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bfa0f9f3251447c922938888ddea917', 'request_id': 'req-f8292090-d26c-430d-84af-f406a05eb1b2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.836657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bfa0f9f3251447c922938888ddea917', 'request_id': 'req-f8292090-d26c-430d-84af-f406a05eb1b2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.849255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] The token cache is being invalidate because project 8bfa0f9f3251447c922938888ddea917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:01:36.849548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.850208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f8292090-d26c-430d-84af-f406a05eb1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:36.852448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1113/2221] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:01:36 2022] DELETE /identity/v3/projects/8bfa0f9f3251447c922938888ddea917 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:01:36.857444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf782f-e8e0-44b7-a9f3-082de766e4db None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:36.857778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf782f-e8e0-44b7-a9f3-082de766e4db None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:36.858137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf782f-e8e0-44b7-a9f3-082de766e4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:36.877343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-dfbf782f-e8e0-44b7-a9f3-082de766e4db None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:36.884472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dfbf782f-e8e0-44b7-a9f3-082de766e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:36.967504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1109/2222] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:36.976383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3ffef-59e1-439f-8096-6df6ba1f9675 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:36.976871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3ffef-59e1-439f-8096-6df6ba1f9675 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:36.977165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3ffef-59e1-439f-8096-6df6ba1f9675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.000496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-1ab3ffef-59e1-439f-8096-6df6ba1f9675 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:37.002211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1ab3ffef-59e1-439f-8096-6df6ba1f9675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:37.017721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1114/2223] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:37.020011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf782f-e8e0-44b7-a9f3-082de766e4db None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:37.033120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf782f-e8e0-44b7-a9f3-082de766e4db None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:37.037911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:37.039686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.040290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.040810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.041902 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:01:37.042450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:37.044829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:37.047408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:37.088072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1110/2224] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:01:37.088793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3ffef-59e1-439f-8096-6df6ba1f9675 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:37.107983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3ffef-59e1-439f-8096-6df6ba1f9675 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:37.113421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:37.114290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.114852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.115460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.117250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:37.119357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:37.153046 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1115/2225] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:01:37 2022] POST /identity/v3/projects => generated 399 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:01:37.158007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:37.179860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97fda4-f343-4330-8cb2-b62bb3cf6998 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:37.307314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:37.308236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.308608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.309031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.310199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:37.312614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:37.357574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1111/2226] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:01:37 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:01:37.359331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:37.370464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad22ae-b53f-47a1-8d96-c38efce58dce admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:37.375054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:37.375722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.376013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.376296 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.377067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:37.378680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:37.389377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1116/2227] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:01:37.393767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:37.404908 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3847c-a883-4722-9904-c9cc0cac57f1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:37.410604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be83b5a-b705-4585-87d9-562f10b6c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:37.411370 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be83b5a-b705-4585-87d9-562f10b6c8ad admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.411776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be83b5a-b705-4585-87d9-562f10b6c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.411940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be83b5a-b705-4585-87d9-562f10b6c8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.412806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be83b5a-b705-4585-87d9-562f10b6c8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:37.414971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be83b5a-b705-4585-87d9-562f10b6c8ad admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:37.425606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1112/2228] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:01:37.430261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:37.440900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da82ce-7ffe-44a0-b865-bff6a7c637f0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:37.447741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:37.449624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.450467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.451163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] PATH_INFO: `/v3/projects/362d6ad46b01415cb0467e45b6592313/users/baeacfaa7a2e46a2bc16f85702863c42/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.473821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=362d6ad46b01415cb0467e45b6592313, user_id=baeacfaa7a2e46a2bc16f85702863c42, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:37.477241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:37.479510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:37.492008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1117/2229] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:01:37 2022] PUT /identity/v3/projects/362d6ad46b01415cb0467e45b6592313/users/baeacfaa7a2e46a2bc16f85702863c42/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:01:37.503046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2dc38-9e5c-4fc8-9856-7614c91f7172 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.503334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2dc38-9e5c-4fc8-9856-7614c91f7172 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.503606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2dc38-9e5c-4fc8-9856-7614c91f7172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.531712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-75c2dc38-9e5c-4fc8-9856-7614c91f7172 None None] MFA Rules not processed for user `baeacfaa7a2e46a2bc16f85702863c42`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:37.532972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-75c2dc38-9e5c-4fc8-9856-7614c91f7172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:37.605429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1113/2230] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:01:37.617534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:37.620492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:37.699401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb117ef-d8b4-4fd9-967e-47c75bf35831 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:37.703389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6fe01-4b6d-4a68-ad01-7432388f692a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:37.704049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6fe01-4b6d-4a68-ad01-7432388f692a service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:37.704332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6fe01-4b6d-4a68-ad01-7432388f692a service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:37.704609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6fe01-4b6d-4a68-ad01-7432388f692a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:37.715027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6fe01-4b6d-4a68-ad01-7432388f692a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:37.716507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6fe01-4b6d-4a68-ad01-7432388f692a service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:37.726033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1118/2231] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:01:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:01:47.602321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:47.721646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:01:51.192023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2dc38-9e5c-4fc8-9856-7614c91f7172 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:51.270955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2dc38-9e5c-4fc8-9856-7614c91f7172 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:51.276348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:51.277295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:51.277577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:51.277853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] PATH_INFO: `/v3/users/baeacfaa7a2e46a2bc16f85702863c42` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:51.279232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baeacfaa7a2e46a2bc16f85702863c42)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:51.281512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:51.333977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baeacfaa7a2e46a2bc16f85702863c42'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:51.334674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baeacfaa7a2e46a2bc16f85702863c42'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:51.343278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baeacfaa7a2e46a2bc16f85702863c42'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:51.344031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baeacfaa7a2e46a2bc16f85702863c42'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:51.353802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baeacfaa7a2e46a2bc16f85702863c42'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:51.379943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1114/2232] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:01:51 2022] DELETE /identity/v3/users/baeacfaa7a2e46a2bc16f85702863c42 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:01:51.385301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:51.385844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:51.385844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:51.405266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:01:51.406542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:51.487270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1119/2233] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:01:51.497724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:51.501926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:51.589920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51626c-31e3-4ccb-af23-2268f74cdf92 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:51.593772 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:51.594480 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:51.594826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:51.595104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:51.605443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:51.606900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:51.618693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1115/2234] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:01:52.243590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:01:52.245725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:01:52.259890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac4981-850e-44d9-b335-71ffbf2ee734 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:01:52.264734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:01:52.265660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:01:52.265963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:01:52.266232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] PATH_INFO: `/v3/projects/362d6ad46b01415cb0467e45b6592313` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:01:52.267480 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=362d6ad46b01415cb0467e45b6592313)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:01:52.271279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:01:52.338183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362d6ad46b01415cb0467e45b6592313', 'request_id': 'req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:52.345963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362d6ad46b01415cb0467e45b6592313', 'request_id': 'req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:52.354807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362d6ad46b01415cb0467e45b6592313', 'request_id': 'req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:52.368676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] The token cache is being invalidate because project 362d6ad46b01415cb0467e45b6592313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:01:52.369019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:52.369719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c2419ea6-5662-4a1f-b53e-dca9a5c0f28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:01:52.371652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1120/2235] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:01:52 2022] DELETE /identity/v3/projects/362d6ad46b01415cb0467e45b6592313 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:02:01.612931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:02:02.364876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:02:08.994622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:08.996751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:09.009076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957380e1-4843-4bf6-8215-352a45a0282d service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:09.060347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:09.061040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:09.061324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:09.061600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:09.063408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:09.135787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:09.137427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:09.149084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1116/2236] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:10.121025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:10.121926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:10.122100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:10.147613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:02:10.149753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:10.229127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1121/2237] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 15:02:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:02:10.260089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:10.261654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:10.335227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:10.337322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:10.356955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead118df-cf95-4ab6-b742-159ed45ace71 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:10.359845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606dc6b-02c9-4f3a-946a-bd5261c20f23 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:10.362533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:10.364008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:10.364511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:10.367710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b27867-2034-4724-8cbf-2fa354d728d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:10.369211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27867-2034-4724-8cbf-2fa354d728d1 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:10.370019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27867-2034-4724-8cbf-2fa354d728d1 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:10.370775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27867-2034-4724-8cbf-2fa354d728d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:10.378193 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:10.380978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:10.398884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:10.400656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b27867-2034-4724-8cbf-2fa354d728d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:10.400832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:10.402680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b27867-2034-4724-8cbf-2fa354d728d1 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:10.416102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1117/2238] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:02:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 15:02:10.420940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1122/2239] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:12.823366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e5db0-958a-4067-a868-f58cc78738de None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:12.825047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e5db0-958a-4067-a868-f58cc78738de None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:12.826127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e5db0-958a-4067-a868-f58cc78738de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:12.866701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-360e5db0-958a-4067-a868-f58cc78738de None None] MFA Rules not processed for user `0b8be0ab53ae48af844bc281dee5f77c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:02:12.868590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-360e5db0-958a-4067-a868-f58cc78738de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:12.910225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:02:12.956609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1118/2240] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:02:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:02:17.087128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:17.126955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5e41f-8c29-4937-969d-3396d6c0aa3e service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:17.137096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:17.138738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:17.139031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:17.141378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:17.146423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:17.293391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:17.296113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:17.329175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1123/2241] 10.222.0.72 () {60 vars in 1396 bytes} [Thu Jun 16 15:02:17 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:18.688557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e5db0-958a-4067-a868-f58cc78738de None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:18.704231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e5db0-958a-4067-a868-f58cc78738de None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:18.708395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:18.709186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:18.709429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:18.710006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:18.711986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:18.792038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:18.793625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:18.817448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1119/2242] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:27.220604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:27.222143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:27.238852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d15c3-1d92-48d0-a51a-c436166c9d5e service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:27.250443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04ebff-004d-4f95-a523-f573add89b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:27.251124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04ebff-004d-4f95-a523-f573add89b38 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:27.251411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04ebff-004d-4f95-a523-f573add89b38 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:27.251692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04ebff-004d-4f95-a523-f573add89b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:27.255028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ca04ebff-004d-4f95-a523-f573add89b38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:27.274071 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04ebff-004d-4f95-a523-f573add89b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:27.275910 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04ebff-004d-4f95-a523-f573add89b38 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:27.286640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1124/2243] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:28.801020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=41 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:02:34.367905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:34.369697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:34.381664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f243f0-580b-458f-a4b3-dfa656209d9f service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:34.385289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbed243-a896-4f7e-8c94-2f9bcacc247d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:34.385929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbed243-a896-4f7e-8c94-2f9bcacc247d service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.386216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbed243-a896-4f7e-8c94-2f9bcacc247d service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.386482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbed243-a896-4f7e-8c94-2f9bcacc247d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.388226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6dbed243-a896-4f7e-8c94-2f9bcacc247d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:34.399405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbed243-a896-4f7e-8c94-2f9bcacc247d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:34.400942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbed243-a896-4f7e-8c94-2f9bcacc247d service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:34.409647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1120/2244] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:34.518746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549955-1f10-4c5c-919b-30d5ab1112e2 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.519322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549955-1f10-4c5c-919b-30d5ab1112e2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.519322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd549955-1f10-4c5c-919b-30d5ab1112e2 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.520866 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1125/2245] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 15:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:02:34.530920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bfef0-070e-4bdf-a98b-f414a60fa1b1 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.531345 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bfef0-070e-4bdf-a98b-f414a60fa1b1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.531605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bfef0-070e-4bdf-a98b-f414a60fa1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.576324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d25bfef0-070e-4bdf-a98b-f414a60fa1b1 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:02:34.577594 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d25bfef0-070e-4bdf-a98b-f414a60fa1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:34.589180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1121/2246] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:02:34.601181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd549955-1f10-4c5c-919b-30d5ab1112e2 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:34.603170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dd549955-1f10-4c5c-919b-30d5ab1112e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:34.609563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=66 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:02:34.686427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd549955-1f10-4c5c-919b-30d5ab1112e2 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:34.692126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e1faa-212a-4b8a-bf3a-94c081318b15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:34.692967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e1faa-212a-4b8a-bf3a-94c081318b15 service glance] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.693350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e1faa-212a-4b8a-bf3a-94c081318b15 service glance] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.694180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e1faa-212a-4b8a-bf3a-94c081318b15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.713733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e1faa-212a-4b8a-bf3a-94c081318b15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:34.715210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e1faa-212a-4b8a-bf3a-94c081318b15 service glance] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:34.726598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1126/2247] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:34.804038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11f41-3194-442b-a61c-ec91338498b1 None None] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.804330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11f41-3194-442b-a61c-ec91338498b1 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.804654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11f41-3194-442b-a61c-ec91338498b1 None None] PATH_INFO: `/` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.808347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1122/2248] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 15:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:02:34.811503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eac4f6-a49f-4dd1-ab68-359056498116 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.811823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eac4f6-a49f-4dd1-ab68-359056498116 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.812075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eac4f6-a49f-4dd1-ab68-359056498116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.831384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f5eac4f6-a49f-4dd1-ab68-359056498116 None None] MFA Rules not processed for user `2fc5135327fe41409cc4ba407b004d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:02:34.832622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f5eac4f6-a49f-4dd1-ab68-359056498116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:34.839579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1127/2249] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:02:34.851706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f11f41-3194-442b-a61c-ec91338498b1 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:34.863357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f11f41-3194-442b-a61c-ec91338498b1 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:34.867736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619874b5-f5ce-4ea2-8bf4-8d690c1f0931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:34.868399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619874b5-f5ce-4ea2-8bf4-8d690c1f0931 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.868885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619874b5-f5ce-4ea2-8bf4-8d690c1f0931 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.869002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619874b5-f5ce-4ea2-8bf4-8d690c1f0931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.877830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619874b5-f5ce-4ea2-8bf4-8d690c1f0931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:34.880615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619874b5-f5ce-4ea2-8bf4-8d690c1f0931 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:34.891207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1123/2250] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:34.986889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811ff34-1b56-49d8-8fe3-01e3c9e42b05 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.987178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811ff34-1b56-49d8-8fe3-01e3c9e42b05 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.987460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811ff34-1b56-49d8-8fe3-01e3c9e42b05 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:34.988868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1128/2251] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 15:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:02:34.995155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:34.995477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:34.995760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:35.014148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] MFA Rules not processed for user `0b8661ae3402445c9c027e739983f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:02:35.015277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:35.024800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1124/2252] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 15:02:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:02:35.036880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811ff34-1b56-49d8-8fe3-01e3c9e42b05 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:35.049935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811ff34-1b56-49d8-8fe3-01e3c9e42b05 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:35.054767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:35.055421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:35.055733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:35.056062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:35.067627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:35.069195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:35.081042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1129/2253] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:35.145893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:35.147356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:35.220580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5319f-e6fe-4f0e-9c24-7d45377790a4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:35.226531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf7c4f-0be9-4b4a-93d6-2085db638df4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:35.227184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7c4f-0be9-4b4a-93d6-2085db638df4 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:35.227464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7c4f-0be9-4b4a-93d6-2085db638df4 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:35.227738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7c4f-0be9-4b4a-93d6-2085db638df4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:35.229565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2ebf7c4f-0be9-4b4a-93d6-2085db638df4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:35.242509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf7c4f-0be9-4b4a-93d6-2085db638df4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:35.244101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf7c4f-0be9-4b4a-93d6-2085db638df4 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:35.255207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1125/2254] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:42.820888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:42.829863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e41e-b4ae-4de8-9403-98a2ea01a0c4 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:42.834368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:42.835114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:42.835469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:42.835866 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:42.837842 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:42.849188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:42.851395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:42.861137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1130/2255] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:02:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:02:43.847377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:43.847970 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:43.848222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:43.868536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:02:43.869800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:02:43.881472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1126/2256] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:02:43.890974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:02:43.903658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ffd62-8f66-4ab6-96be-283f2f5f7e9f service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:02:43.911176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc803c-58ed-4c8d-bf66-3e1cc6b7d7fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:02:43.912018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc803c-58ed-4c8d-bf66-3e1cc6b7d7fb service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:02:43.912389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc803c-58ed-4c8d-bf66-3e1cc6b7d7fb service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:02:43.912726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc803c-58ed-4c8d-bf66-3e1cc6b7d7fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:02:43.923976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc803c-58ed-4c8d-bf66-3e1cc6b7d7fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:02:43.926055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc803c-58ed-4c8d-bf66-3e1cc6b7d7fb service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:02:43.936248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1131/2257] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:02:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:02:53.861748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=59 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:02:53.932135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=46 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:13.891822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:13.893886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:13.915277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a5ee8-1671-4186-9fdd-3b47461a4e16 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:13.924928 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:13.926611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:13.927061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:13.927572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:13.931373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:13.958266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:03:13.960514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:03:13.973515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1127/2258] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:03:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:03:17.180160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c045-5bf7-4d97-8455-7715d61e97e6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:17.180711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c045-5bf7-4d97-8455-7715d61e97e6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:17.180825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c045-5bf7-4d97-8455-7715d61e97e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:17.202636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-e0e3c045-5bf7-4d97-8455-7715d61e97e6 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:03:17.203861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e0e3c045-5bf7-4d97-8455-7715d61e97e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:17.212221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1132/2259] 10.222.0.72 () {58 vars in 967 bytes} [Thu Jun 16 15:03:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:03:17.239489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:17.251029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301a338-9de2-4a0c-9865-84a86bb7e03a service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:17.256302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:17.257028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:17.257310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:17.257583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:17.266398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:03:17.267988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:03:17.281479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1128/2260] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:03:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:03:27.196989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:27.273156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=12 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:03:53.502752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c045-5bf7-4d97-8455-7715d61e97e6 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:03:53.515685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c045-5bf7-4d97-8455-7715d61e97e6 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:03:53.521937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6146fd7-18d6-47cf-8ef5-118eaa7118d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:03:53.522594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146fd7-18d6-47cf-8ef5-118eaa7118d4 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:03:53.522879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146fd7-18d6-47cf-8ef5-118eaa7118d4 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:03:53.523159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6146fd7-18d6-47cf-8ef5-118eaa7118d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:03:53.525014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e6146fd7-18d6-47cf-8ef5-118eaa7118d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:03:53.583539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6146fd7-18d6-47cf-8ef5-118eaa7118d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:03:53.585177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6146fd7-18d6-47cf-8ef5-118eaa7118d4 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:03:53.594401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1133/2261] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:03:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:03.590035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=28 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:09.914156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:09.927555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe64c767-0d87-485b-b9f7-51d16586bb42 service swift] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:09.934055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:09.934837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:09.935216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:09.935573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:09.937622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:09.951063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:09.952794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:09.965374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1129/2262] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:11.165660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5dde-f117-44c8-a466-702ec0f124c8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:11.166220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5dde-f117-44c8-a466-702ec0f124c8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:11.166380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5dde-f117-44c8-a466-702ec0f124c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:11.190321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-073a5dde-f117-44c8-a466-702ec0f124c8 None None] MFA Rules not processed for user `9d977f609537476daa8fe8adbab9ff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:11.191851 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-073a5dde-f117-44c8-a466-702ec0f124c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:11.255878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1134/2263] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:04:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:11.581852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:11.591677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70de45c-15a7-42a6-a9e2-f6899f142791 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:11.595527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:11.596188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:11.596473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:11.596750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:11.598574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:11.608982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:11.610414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:11.619076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1130/2264] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:21.248872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=25 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:21.614887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=12 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:23.334303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a51bc-e110-431d-8c77-ec290982719e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:23.334878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a51bc-e110-431d-8c77-ec290982719e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:23.334878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a51bc-e110-431d-8c77-ec290982719e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:23.352907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5a7a51bc-e110-431d-8c77-ec290982719e None None] MFA Rules not processed for user `73a51a4e876e4910bbc5dd69f8a18058`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:23.354124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5a7a51bc-e110-431d-8c77-ec290982719e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:23.361130 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1135/2265] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:31.886049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:31.900966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411ebb7-04b6-4d59-9556-56a573ab50ed service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:31.910310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32178fbb-7704-4116-b85f-72b62d536348 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:31.910990 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32178fbb-7704-4116-b85f-72b62d536348 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:31.911277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32178fbb-7704-4116-b85f-72b62d536348 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:31.911620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32178fbb-7704-4116-b85f-72b62d536348 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:31.913561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-32178fbb-7704-4116-b85f-72b62d536348 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:31.928496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32178fbb-7704-4116-b85f-72b62d536348 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:31.930204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32178fbb-7704-4116-b85f-72b62d536348 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:31.941955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1131/2266] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:33.347543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:34.600181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a51bc-e110-431d-8c77-ec290982719e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:04:34.614885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a51bc-e110-431d-8c77-ec290982719e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:04:34.621506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e8303-e697-43e1-be48-68c13e63baf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:04:34.622367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e8303-e697-43e1-be48-68c13e63baf2 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:34.622832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e8303-e697-43e1-be48-68c13e63baf2 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:34.623263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e8303-e697-43e1-be48-68c13e63baf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:34.625387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5c2e8303-e697-43e1-be48-68c13e63baf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:34.709393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e8303-e697-43e1-be48-68c13e63baf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:04:34.711192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2e8303-e697-43e1-be48-68c13e63baf2 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:04:34.721374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1136/2267] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:04:41.937152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:44.716383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=28 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:48.872186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:48.872652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:48.872760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:48.891600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] MFA Rules not processed for user `0b8be0ab53ae48af844bc281dee5f77c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:48.892823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:48.900791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1132/2268] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:49.257249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e69475-7738-4769-87bb-1e991077d226 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:04:49.257813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e69475-7738-4769-87bb-1e991077d226 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:04:49.258700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e69475-7738-4769-87bb-1e991077d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:04:49.276503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-10e69475-7738-4769-87bb-1e991077d226 None None] MFA Rules not processed for user `ba40bcf173404e6c8b17d311b65bf780`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:04:49.277677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-10e69475-7738-4769-87bb-1e991077d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:04:49.284800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1137/2269] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:04:58.886458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:04:59.271297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:05:48.248112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:48.249536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:48.263865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f52a0-854f-4683-a268-e63480ac3aa0 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:48.269064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:48.269710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:48.269999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:48.270275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:48.272002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:48.284725 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:48.286354 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:48.296921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1133/2270] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:49.991614 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677d4a2-4ab0-4c72-a3c5-443d708aa105 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:49.992110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677d4a2-4ab0-4c72-a3c5-443d708aa105 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:49.992223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677d4a2-4ab0-4c72-a3c5-443d708aa105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:50.009692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2677d4a2-4ab0-4c72-a3c5-443d708aa105 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:50.010875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2677d4a2-4ab0-4c72-a3c5-443d708aa105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:50.018534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1138/2271] 10.222.0.72 () {58 vars in 967 bytes} [Thu Jun 16 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:50.065280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:50.076684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a0772-8075-4d21-bb2c-0edb348a9aa8 service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:50.082130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddc96e-f198-4294-9b97-df7b49b54dc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:50.082800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddc96e-f198-4294-9b97-df7b49b54dc7 service swift] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:50.083088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddc96e-f198-4294-9b97-df7b49b54dc7 service swift] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:50.083368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddc96e-f198-4294-9b97-df7b49b54dc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:50.092156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfddc96e-f198-4294-9b97-df7b49b54dc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:50.093775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfddc96e-f198-4294-9b97-df7b49b54dc7 service swift] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:50.102036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1134/2272] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:05:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:05:53.640281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cb52-303b-4df8-b714-83bf0bf308e6 None None] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:53.641181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cb52-303b-4df8-b714-83bf0bf308e6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:53.641526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898cb52-303b-4df8-b714-83bf0bf308e6 None None] PATH_INFO: `/` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:53.643100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1139/2273] 10.222.0.72 () {54 vars in 921 bytes} [Thu Jun 16 15:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:05:53.656596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a84ee-afe8-4782-afff-a9b24c0d0190 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:53.657090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a84ee-afe8-4782-afff-a9b24c0d0190 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:53.657416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a84ee-afe8-4782-afff-a9b24c0d0190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:53.676962 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-260a84ee-afe8-4782-afff-a9b24c0d0190 None None] MFA Rules not processed for user `848d858044ed43e69aa055491cc29d80`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:05:53.678310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-260a84ee-afe8-4782-afff-a9b24c0d0190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:53.687918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1135/2274] 10.222.0.72 () {58 vars in 1018 bytes} [Thu Jun 16 15:05:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:05:53.715637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898cb52-303b-4df8-b714-83bf0bf308e6 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:53.739776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898cb52-303b-4df8-b714-83bf0bf308e6 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:53.750276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:53.750946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:53.751242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:53.751523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:53.760598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:53.762820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:53.772852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1140/2275] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:55.419395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a84ee-afe8-4782-afff-a9b24c0d0190 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:55.428558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a84ee-afe8-4782-afff-a9b24c0d0190 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:55.432130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:55.432773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:55.433658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:55.433944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:55.435708 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:55.503120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:55.504483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:55.514899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1136/2276] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:05:57.272069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:57.274014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:57.311289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031211ac-cfa4-4b24-bda6-a9b470978d47 service placement] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:57.327976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:57.329498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:57.329847 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:57.330477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] PATH_INFO: `/v3/users/d0b498bcfc434db99010657e2ab544b9` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:57.332742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b498bcfc434db99010657e2ab544b9)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:57.335380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:57.449827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b498bcfc434db99010657e2ab544b9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.450943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b498bcfc434db99010657e2ab544b9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.452478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b498bcfc434db99010657e2ab544b9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.462108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b498bcfc434db99010657e2ab544b9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.490610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b498bcfc434db99010657e2ab544b9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.523619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1141/2277] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:05:57 2022] DELETE /identity/v3/users/d0b498bcfc434db99010657e2ab544b9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:05:57.706886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:57.719550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159dac0-d69c-4162-b242-e994188182d7 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:57.783894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:57.785147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:57.785898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:57.786369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] PATH_INFO: `/v3/projects/3048a9bbedf74d49a27c9911ca2ad960` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:57.788184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3048a9bbedf74d49a27c9911ca2ad960)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:57.790471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:57.847658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3048a9bbedf74d49a27c9911ca2ad960', 'request_id': 'req-9891a180-5818-48a4-970a-7f3d6cfaa7b3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.855203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3048a9bbedf74d49a27c9911ca2ad960', 'request_id': 'req-9891a180-5818-48a4-970a-7f3d6cfaa7b3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.861108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:05:57.868884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3048a9bbedf74d49a27c9911ca2ad960', 'request_id': 'req-9891a180-5818-48a4-970a-7f3d6cfaa7b3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.886955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] The token cache is being invalidate because project 3048a9bbedf74d49a27c9911ca2ad960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:05:57.887366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.888054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:57.889965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1137/2278] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:05:57 2022] DELETE /identity/v3/projects/3048a9bbedf74d49a27c9911ca2ad960 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:05:57.894280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:57.895982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:57.911396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a8abc-575c-43ec-b682-91f35763d0a9 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:57.974865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:57.975749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:57.976032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:57.976306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] PATH_INFO: `/v3/users/35195eb3115f408ba9ab4c9a9d123570` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:57.977543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] RBAC: Authorizing `identity:delete_user(user_id=35195eb3115f408ba9ab4c9a9d123570)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:57.979486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:58.029153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35195eb3115f408ba9ab4c9a9d123570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.029840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35195eb3115f408ba9ab4c9a9d123570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.030488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35195eb3115f408ba9ab4c9a9d123570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.036363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:05:58.038409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35195eb3115f408ba9ab4c9a9d123570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.052627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35195eb3115f408ba9ab4c9a9d123570'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.067301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1142/2279] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:05:57 2022] DELETE /identity/v3/users/35195eb3115f408ba9ab4c9a9d123570 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:05:58.247375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:05:58.249270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:05:58.266651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891a180-5818-48a4-970a-7f3d6cfaa7b3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:05:58.327600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:05:58.328483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:05:58.328776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:05:58.329078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] PATH_INFO: `/v3/projects/c732e283521445d58a8d0445bc7d6759` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:05:58.330289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c732e283521445d58a8d0445bc7d6759)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:05:58.332223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:05:58.382870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c732e283521445d58a8d0445bc7d6759', 'request_id': 'req-29777b84-fc01-421d-b738-168838ce5e6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.388441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c732e283521445d58a8d0445bc7d6759', 'request_id': 'req-29777b84-fc01-421d-b738-168838ce5e6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.396209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c732e283521445d58a8d0445bc7d6759', 'request_id': 'req-29777b84-fc01-421d-b738-168838ce5e6d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.408607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] The token cache is being invalidate because project c732e283521445d58a8d0445bc7d6759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:05:58.408940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.409633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:05:58.411181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1138/2280] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:05:58 2022] DELETE /identity/v3/projects/c732e283521445d58a8d0445bc7d6759 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:06.795707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:06.797140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:06.854354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba22582-2b85-4a12-af82-d223ae82293e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:06.905451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971867-2696-41db-b029-8129de014260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:06.906697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971867-2696-41db-b029-8129de014260 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:06.907130 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971867-2696-41db-b029-8129de014260 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:06.907462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971867-2696-41db-b029-8129de014260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:06.909376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4b971867-2696-41db-b029-8129de014260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:06.973321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b971867-2696-41db-b029-8129de014260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:06.974766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b971867-2696-41db-b029-8129de014260 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:06.984106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1143/2281] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:08.404982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:10.052111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.053645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:10.105311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971867-2696-41db-b029-8129de014260 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.106981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4b971867-2696-41db-b029-8129de014260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:10.121441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971867-2696-41db-b029-8129de014260 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.126026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.127464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.127758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.128038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] PATH_INFO: `/v3/users/73a51a4e876e4910bbc5dd69f8a18058` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.129387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a51a4e876e4910bbc5dd69f8a18058)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.129874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29777b84-fc01-421d-b738-168838ce5e6d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.131506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.143143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.143143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.143143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.143143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.144650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:10.158393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.160797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.170852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1139/2282] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:10.180025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:10.196305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a51a4e876e4910bbc5dd69f8a18058'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.196991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a51a4e876e4910bbc5dd69f8a18058'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.197703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a51a4e876e4910bbc5dd69f8a18058'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.205145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a51a4e876e4910bbc5dd69f8a18058'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.217479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-195b2375-b256-4d41-95bd-192cf37673be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a51a4e876e4910bbc5dd69f8a18058'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.229897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1144/2283] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:10 2022] DELETE /identity/v3/users/73a51a4e876e4910bbc5dd69f8a18058 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:10.409814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.421831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f41f7b-bf90-4afd-bba6-780796a64d01 service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.480898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.481750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.482030 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.482302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] PATH_INFO: `/v3/projects/b017c2881aba4df6afe70cdcb84d5ee8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.483467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b017c2881aba4df6afe70cdcb84d5ee8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.485381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.531019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b017c2881aba4df6afe70cdcb84d5ee8', 'request_id': 'req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.536609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b017c2881aba4df6afe70cdcb84d5ee8', 'request_id': 'req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.544131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b017c2881aba4df6afe70cdcb84d5ee8', 'request_id': 'req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.555276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] The token cache is being invalidate because project b017c2881aba4df6afe70cdcb84d5ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:10.555561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.556187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8d0f2ab-2f24-490c-9f97-c1ca4ce3aca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:10.558335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1140/2284] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:10 2022] DELETE /identity/v3/projects/b017c2881aba4df6afe70cdcb84d5ee8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:10.563205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019908c-ca09-4078-823d-5a84c8658a4d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.563503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019908c-ca09-4078-823d-5a84c8658a4d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.563805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1019908c-ca09-4078-823d-5a84c8658a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.582137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-1019908c-ca09-4078-823d-5a84c8658a4d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:10.583282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1019908c-ca09-4078-823d-5a84c8658a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:10.643552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1145/2285] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:10.651542 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265e970-0f4f-43b6-bdda-c36059413a81 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.651827 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265e970-0f4f-43b6-bdda-c36059413a81 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.652104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265e970-0f4f-43b6-bdda-c36059413a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.671277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3265e970-0f4f-43b6-bdda-c36059413a81 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:10.672494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3265e970-0f4f-43b6-bdda-c36059413a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:10.680602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1141/2286] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:10.685649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019908c-ca09-4078-823d-5a84c8658a4d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.695317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1019908c-ca09-4078-823d-5a84c8658a4d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.699839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.700485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.700768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.701076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.701798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:10.702110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.703702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.715057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1146/2287] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:06:10.719237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265e970-0f4f-43b6-bdda-c36059413a81 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.728239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265e970-0f4f-43b6-bdda-c36059413a81 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.733346 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.733983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.734262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.734532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.735222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.737123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.747173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:10.772859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1142/2288] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:10 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:10.777406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.787802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e97385-5e89-49fc-8cb4-604e0c56141a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.844295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.844912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.845207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.845489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.846186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.848122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.868110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1147/2289] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:10 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:10.872160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.882398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59a71b-a40a-4ca5-8bc8-f40582a09d70 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.887047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.887711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.887998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.888269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.889035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.890623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.900493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1143/2290] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:10.904818 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.914941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d0495-ffac-4a7e-9987-1a3b2a7b9b33 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.920136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc2932-82e2-430c-9788-6959a5532c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.920783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2932-82e2-430c-9788-6959a5532c48 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.921097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2932-82e2-430c-9788-6959a5532c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.921375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2932-82e2-430c-9788-6959a5532c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.922108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc2932-82e2-430c-9788-6959a5532c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.923697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc2932-82e2-430c-9788-6959a5532c48 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.933527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1148/2291] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:10.937436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:10.947368 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489110d-df65-4762-bdcc-7dc5aded8955 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:10.952160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:10.953078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.953366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.953647 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] PATH_INFO: `/v3/projects/098974f73d494ffa9382069577b02a17/users/62fa2bedd01d4f62bc86227673a71fa7/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:10.968690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=098974f73d494ffa9382069577b02a17, user_id=62fa2bedd01d4f62bc86227673a71fa7, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:10.971393 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:10.985525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1144/2292] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:10 2022] PUT /identity/v3/projects/098974f73d494ffa9382069577b02a17/users/62fa2bedd01d4f62bc86227673a71fa7/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:10.996569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d764da-69cf-48ef-8daf-d7f5d0a53a83 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:10.996891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d764da-69cf-48ef-8daf-d7f5d0a53a83 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:10.997181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d764da-69cf-48ef-8daf-d7f5d0a53a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.024082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-10d764da-69cf-48ef-8daf-d7f5d0a53a83 None None] MFA Rules not processed for user `62fa2bedd01d4f62bc86227673a71fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:11.025309 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-10d764da-69cf-48ef-8daf-d7f5d0a53a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:11.029733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:11.092473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1149/2293] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:11.096900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:11.159058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c623c8b-3407-4ff3-a9ff-08e08cc6a576 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:11.163770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:11.164428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.164745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.165054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.165761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:11.168173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:11.193619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1145/2294] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:11 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:06:11.197431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d764da-69cf-48ef-8daf-d7f5d0a53a83 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:11.208154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d764da-69cf-48ef-8daf-d7f5d0a53a83 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:11.264169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:11.264857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.265145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.265457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.266198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:11.268230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:11.288184 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1150/2295] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:11 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:11.292635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:11.303151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11823f6a-9ee5-4940-b6f4-6ec2ea318aaa admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:11.308361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:11.309048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.309337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.309613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.310336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:11.312014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:11.322569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1146/2296] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:11.326573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:11.336606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671865dc-f6af-415c-a4ec-5e3553d616c8 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:11.341251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:11.342155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.342441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.342723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] PATH_INFO: `/v3/projects/31d3702c05e541a8babe0e7277d2d61a/users/bbf683d57f154a23916e9706fa9f8e8d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.357809 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d3702c05e541a8babe0e7277d2d61a, user_id=bbf683d57f154a23916e9706fa9f8e8d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:11.360424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:11.374313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1151/2297] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:11 2022] PUT /identity/v3/projects/31d3702c05e541a8babe0e7277d2d61a/users/bbf683d57f154a23916e9706fa9f8e8d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:11.378228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:11.388227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543f466-5113-4bd5-a5be-edd58b377bda admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:11.443442 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c298b-3807-4e1c-9794-a162ecb19fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:11.444144 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c298b-3807-4e1c-9794-a162ecb19fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.444429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c298b-3807-4e1c-9794-a162ecb19fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.444707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c298b-3807-4e1c-9794-a162ecb19fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.445469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c298b-3807-4e1c-9794-a162ecb19fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:11.447060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c298b-3807-4e1c-9794-a162ecb19fe2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:11.456929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1147/2298] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:11.461162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:11.471931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725ff17-1c1f-4a70-bd37-879b35b16e62 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:11.476755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:11.477674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.477962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.478239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] PATH_INFO: `/v3/projects/31d3702c05e541a8babe0e7277d2d61a/users/bbf683d57f154a23916e9706fa9f8e8d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.480735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d3702c05e541a8babe0e7277d2d61a, user_id=bbf683d57f154a23916e9706fa9f8e8d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:11.483420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:11.493588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1152/2299] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:11 2022] PUT /identity/v3/projects/31d3702c05e541a8babe0e7277d2d61a/users/bbf683d57f154a23916e9706fa9f8e8d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:11.504019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bf226-b6c5-40ae-85cf-0ddee656b728 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.504313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bf226-b6c5-40ae-85cf-0ddee656b728 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.504591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bf226-b6c5-40ae-85cf-0ddee656b728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.532278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f94bf226-b6c5-40ae-85cf-0ddee656b728 None None] MFA Rules not processed for user `bbf683d57f154a23916e9706fa9f8e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:11.533527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f94bf226-b6c5-40ae-85cf-0ddee656b728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:11.538544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:11.601895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1148/2300] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:11.614375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:11.616082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:11.672353 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc07690-668f-4ff0-bf58-7f77713a7406 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:11.675999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:11.676631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:11.676947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:11.677228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:11.686040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:11.687493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:11.692377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:11.697618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1153/2301] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:12.000952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bf226-b6c5-40ae-85cf-0ddee656b728 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:12.010178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bf226-b6c5-40ae-85cf-0ddee656b728 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:12.015930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbf683d57f154a23916e9706fa9f8e8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31d3702c05e541a8babe0e7277d2d61a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:12.016582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:12.016899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:12.017186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:12.017915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:12.018228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:12.019825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:12.032282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1149/2302] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:06:12.036259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:12.045310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f70a-87f8-4754-bdc5-1ce80cd95915 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:12.049838 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbf683d57f154a23916e9706fa9f8e8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31d3702c05e541a8babe0e7277d2d61a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:12.050484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:12.050780 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:12.051061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:12.051749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:12.053736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:12.079020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1154/2303] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:12 2022] POST /identity/v3/projects => generated 355 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:12.083067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:12.093538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81354c2a-67a6-439e-899d-7ef71ba98788 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:12.149333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbf683d57f154a23916e9706fa9f8e8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31d3702c05e541a8babe0e7277d2d61a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:12.149958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:12.150201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:12.150485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:12.151738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:12.152060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:12.153683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:12.165140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1150/2304] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:06:12.202124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:12.212153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f60583-ac9f-442b-8a60-a5df6cbf184d tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:12.217336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbf683d57f154a23916e9706fa9f8e8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31d3702c05e541a8babe0e7277d2d61a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:12.218182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:12.218473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:12.218751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] PATH_INFO: `/v3/projects/3522154c02e24b728092f27c5827abf3` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:12.219920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3522154c02e24b728092f27c5827abf3)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:12.221868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:12.268697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3522154c02e24b728092f27c5827abf3', 'request_id': 'req-82c855c1-6b6b-479d-baea-bd1c8de7358f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.274125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3522154c02e24b728092f27c5827abf3', 'request_id': 'req-82c855c1-6b6b-479d-baea-bd1c8de7358f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.281947 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3522154c02e24b728092f27c5827abf3', 'request_id': 'req-82c855c1-6b6b-479d-baea-bd1c8de7358f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.294315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] The token cache is being invalidate because project 3522154c02e24b728092f27c5827abf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:12.294610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.295269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-82c855c1-6b6b-479d-baea-bd1c8de7358f tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.296817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1155/2305] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:12 2022] DELETE /identity/v3/projects/3522154c02e24b728092f27c5827abf3 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:12.482191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:12.483979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:12.550626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dcaac-857a-4e1a-8878-aefdb1ea8ae7 tempest-VolumeQuotaClassesTest-1020483801 tempest-VolumeQuotaClassesTest-1020483801-project-admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:12.580321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:12.608822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:12.609709 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:12.609991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:12.610292 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] PATH_INFO: `/v3/users/62fa2bedd01d4f62bc86227673a71fa7` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:12.611468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fa2bedd01d4f62bc86227673a71fa7)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:12.613428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:12.657389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fa2bedd01d4f62bc86227673a71fa7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.657973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fa2bedd01d4f62bc86227673a71fa7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.665350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fa2bedd01d4f62bc86227673a71fa7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.665987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fa2bedd01d4f62bc86227673a71fa7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.673733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fa2bedd01d4f62bc86227673a71fa7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:12.690970 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1151/2306] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:12 2022] DELETE /identity/v3/users/62fa2bedd01d4f62bc86227673a71fa7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:12.695416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602121a-d816-4430-b753-1123eed98b25 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:12.695714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602121a-d816-4430-b753-1123eed98b25 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:12.696002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602121a-d816-4430-b753-1123eed98b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:12.715917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7602121a-d816-4430-b753-1123eed98b25 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:12.717174 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7602121a-d816-4430-b753-1123eed98b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:12.773791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1156/2307] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:12.784163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:12.785577 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:12.841352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b660a41-822c-407b-bb67-26165e62190c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:12.844937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:12.845574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:12.845853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:12.846125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:12.854922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:12.856327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:12.865605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1152/2308] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:13.127472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602121a-d816-4430-b753-1123eed98b25 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:13.129056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7602121a-d816-4430-b753-1123eed98b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:13.141904 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602121a-d816-4430-b753-1123eed98b25 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:13.146564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:13.147432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:13.147711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:13.147986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] PATH_INFO: `/v3/projects/098974f73d494ffa9382069577b02a17` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:13.149246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=098974f73d494ffa9382069577b02a17)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:13.151222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:13.199065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098974f73d494ffa9382069577b02a17', 'request_id': 'req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.203913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:13.205218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098974f73d494ffa9382069577b02a17', 'request_id': 'req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.214101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098974f73d494ffa9382069577b02a17', 'request_id': 'req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.225776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] The token cache is being invalidate because project 098974f73d494ffa9382069577b02a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:13.226095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.226727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.228345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1157/2309] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:13 2022] DELETE /identity/v3/projects/098974f73d494ffa9382069577b02a17 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:13.232619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:13.242257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e77f52-fd63-45f8-aa50-63a6ab62a9a9 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:13.289630 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:13.299737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:13.300606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:13.300924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:13.301202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] PATH_INFO: `/v3/users/bbf683d57f154a23916e9706fa9f8e8d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:13.302388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf683d57f154a23916e9706fa9f8e8d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:13.304302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:13.349615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf683d57f154a23916e9706fa9f8e8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.350253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf683d57f154a23916e9706fa9f8e8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.357036 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf683d57f154a23916e9706fa9f8e8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.357680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf683d57f154a23916e9706fa9f8e8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.365090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-db01d812-be42-4978-8443-c33e8f160509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf683d57f154a23916e9706fa9f8e8d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.383383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1153/2310] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:13 2022] DELETE /identity/v3/users/bbf683d57f154a23916e9706fa9f8e8d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:13.659009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:13.660499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:13.678409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbd957-3a34-4fe4-8c36-bcdb6ab12981 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:13.736478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:13.737315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:13.737606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:13.737883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] PATH_INFO: `/v3/projects/31d3702c05e541a8babe0e7277d2d61a` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:13.739069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d3702c05e541a8babe0e7277d2d61a)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:13.741076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:13.787501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d3702c05e541a8babe0e7277d2d61a', 'request_id': 'req-4fdd3216-51b7-44b0-b62b-bcb063afbec6'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.793057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d3702c05e541a8babe0e7277d2d61a', 'request_id': 'req-4fdd3216-51b7-44b0-b62b-bcb063afbec6'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.800428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d3702c05e541a8babe0e7277d2d61a', 'request_id': 'req-4fdd3216-51b7-44b0-b62b-bcb063afbec6'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.812082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] The token cache is being invalidate because project 31d3702c05e541a8babe0e7277d2d61a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:13.812375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.813036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4fdd3216-51b7-44b0-b62b-bcb063afbec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:13.814572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1158/2311] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:13 2022] DELETE /identity/v3/projects/31d3702c05e541a8babe0e7277d2d61a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:13.819461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5910be0-8600-4bb0-bdb4-a1e17e3f6d4b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:13.819749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5910be0-8600-4bb0-bdb4-a1e17e3f6d4b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:13.820025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5910be0-8600-4bb0-bdb4-a1e17e3f6d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:13.837938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f5910be0-8600-4bb0-bdb4-a1e17e3f6d4b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:13.839157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f5910be0-8600-4bb0-bdb4-a1e17e3f6d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:13.903516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1154/2312] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:13.911779 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90103679-8b88-4bc4-8e94-3eba07ef012b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:13.912073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90103679-8b88-4bc4-8e94-3eba07ef012b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:13.912356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90103679-8b88-4bc4-8e94-3eba07ef012b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:13.930478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-90103679-8b88-4bc4-8e94-3eba07ef012b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:13.931684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-90103679-8b88-4bc4-8e94-3eba07ef012b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:13.940069 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1159/2313] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:13.944324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5910be0-8600-4bb0-bdb4-a1e17e3f6d4b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:13.954766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5910be0-8600-4bb0-bdb4-a1e17e3f6d4b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:13.959380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:13.960034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:13.960320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:13.960601 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:13.961371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:13.961684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:13.963287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:13.974753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1155/2314] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:06:13.978757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90103679-8b88-4bc4-8e94-3eba07ef012b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:13.987877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90103679-8b88-4bc4-8e94-3eba07ef012b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:13.992414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:13.993103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:13.993391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:13.993673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:13.994406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:13.996823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.022168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1160/2315] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:13 2022] POST /identity/v3/projects => generated 393 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:14.026248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.036361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590ae03-2ace-41dd-adfa-123e34a2ab4c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.092386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.093083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.093363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.093637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.094347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.096253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.102688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:14.116230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1156/2316] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:14 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:14.121090 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.131139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e0e60-8e8b-487a-a520-8cf01fcb79db admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.136293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.137021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.137305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.137607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.138365 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.139963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.150326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1161/2317] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:14.154305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.164504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf53ca-d6c5-4227-8c1f-d3005b34ee18 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.169689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f6ea-0dcc-4a0d-86c3-70306c864fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.170444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f6ea-0dcc-4a0d-86c3-70306c864fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.170729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f6ea-0dcc-4a0d-86c3-70306c864fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.171004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f6ea-0dcc-4a0d-86c3-70306c864fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.171730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f6ea-0dcc-4a0d-86c3-70306c864fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.173353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f6ea-0dcc-4a0d-86c3-70306c864fd2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.183792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1157/2318] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:14.187934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.197795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3e2f6-090e-4e9c-a0d2-6cbfe7c1fc9e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.202355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.203228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.203511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.203784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] PATH_INFO: `/v3/projects/478aeac0de3b4c3d82b0a8974d2f258e/users/7d590ece167d4ddabab8e4723d1c7120/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.218026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478aeac0de3b4c3d82b0a8974d2f258e, user_id=7d590ece167d4ddabab8e4723d1c7120, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.220721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.234194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1162/2319] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:14 2022] PUT /identity/v3/projects/478aeac0de3b4c3d82b0a8974d2f258e/users/7d590ece167d4ddabab8e4723d1c7120/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:14.246256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d114-f981-4ada-afad-49fc69230679 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.246718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d114-f981-4ada-afad-49fc69230679 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.246833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70d114-f981-4ada-afad-49fc69230679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.272748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ff70d114-f981-4ada-afad-49fc69230679 None None] MFA Rules not processed for user `7d590ece167d4ddabab8e4723d1c7120`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:14.274016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ff70d114-f981-4ada-afad-49fc69230679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:14.337603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1158/2320] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:14.342077 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.355795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:14.415986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85866f3-be6e-44f2-ba9b-28265d7db106 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.421475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.422138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.422424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.422732 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.423431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.425399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.450563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1163/2321] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:14 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:14.454665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70d114-f981-4ada-afad-49fc69230679 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.465627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70d114-f981-4ada-afad-49fc69230679 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.526849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.527568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.527891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.528156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.528859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.530885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.553659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1159/2322] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:14 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:14.557663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.568616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6738-e701-4458-8d27-d0e8543c42be admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.574304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.574996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.575273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.575552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.576287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.578008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.592343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1164/2323] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:14.593559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.603895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956da7c9-fc03-4342-821d-172c5486e7cc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.609186 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.610071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.610352 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.610629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] PATH_INFO: `/v3/projects/f8cb39ecda4c4585a5df865488513f51/users/5bfce2feb8934fe69c37c490e9bec014/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.625440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cb39ecda4c4585a5df865488513f51, user_id=5bfce2feb8934fe69c37c490e9bec014, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.628109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.641496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1160/2324] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:14 2022] PUT /identity/v3/projects/f8cb39ecda4c4585a5df865488513f51/users/5bfce2feb8934fe69c37c490e9bec014/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:14.645409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.656079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd527ccc-d477-4723-b194-770629fcb446 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.714188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67184802-e889-4586-9a61-4dd9d54dc9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.714869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67184802-e889-4586-9a61-4dd9d54dc9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.715171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67184802-e889-4586-9a61-4dd9d54dc9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.715435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67184802-e889-4586-9a61-4dd9d54dc9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.716176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67184802-e889-4586-9a61-4dd9d54dc9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.717801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67184802-e889-4586-9a61-4dd9d54dc9a6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.728157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1165/2325] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:14.732971 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.743282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e266411-e102-4114-ac4c-16896e92d16d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.747915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.748816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.749108 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.749385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] PATH_INFO: `/v3/projects/f8cb39ecda4c4585a5df865488513f51/users/5bfce2feb8934fe69c37c490e9bec014/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.751351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cb39ecda4c4585a5df865488513f51, user_id=5bfce2feb8934fe69c37c490e9bec014, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.754081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.765074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1161/2326] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:14 2022] PUT /identity/v3/projects/f8cb39ecda4c4585a5df865488513f51/users/5bfce2feb8934fe69c37c490e9bec014/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:14.775131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.775496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.775843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.803549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] MFA Rules not processed for user `5bfce2feb8934fe69c37c490e9bec014`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:14.804800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:14.875076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1166/2327] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:14.889396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:14.891088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:14.918999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:14.957555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae647615-ca37-4b01-a63a-32ab1cf7a4e3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:14.961214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:14.961866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:14.962149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:14.962429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:14.972376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:14.974071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:14.983642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1162/2328] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:15.032982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:15.034464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:15.046436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbba0ee-cb1a-4a4b-8b22-5c2ec77068d5 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:15.050164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db4786-1997-45ba-9fec-b4a8857eba7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:15.050819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4786-1997-45ba-9fec-b4a8857eba7d service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:15.051105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4786-1997-45ba-9fec-b4a8857eba7d service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:15.051386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4786-1997-45ba-9fec-b4a8857eba7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:15.055872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:15.111883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db4786-1997-45ba-9fec-b4a8857eba7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:15.113392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db4786-1997-45ba-9fec-b4a8857eba7d service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:15.122565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1167/2329] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:19.427106 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:19.492246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df3014-1da2-4d60-8465-807d8da19216 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:19.496701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:19.497585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:19.497893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:19.498156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] PATH_INFO: `/v3/users/7d590ece167d4ddabab8e4723d1c7120` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:19.499329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d590ece167d4ddabab8e4723d1c7120)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:19.501310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:19.554158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d590ece167d4ddabab8e4723d1c7120'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:19.554940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d590ece167d4ddabab8e4723d1c7120'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:19.562328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d590ece167d4ddabab8e4723d1c7120'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:19.562995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d590ece167d4ddabab8e4723d1c7120'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:19.570291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d590ece167d4ddabab8e4723d1c7120'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:19.588188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1163/2330] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:19 2022] DELETE /identity/v3/users/7d590ece167d4ddabab8e4723d1c7120 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:19.592787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:19.593299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:19.593397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:19.613739 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:19.615060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:19.679569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1168/2331] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:19.689892 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:19.691381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:19.737689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:19.751509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c138ba3-551a-40ae-8f18-4f8847fc0e6d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:19.755319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:19.756006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:19.756293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:19.756581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:19.766627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:19.768130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:19.778695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1164/2332] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:20.164805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:20.166426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:20.179393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f893e-88e1-443b-b4fb-396bd32cfac9 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:20.185697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:20.186782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:20.187740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:20.188015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] PATH_INFO: `/v3/projects/478aeac0de3b4c3d82b0a8974d2f258e` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:20.189397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=478aeac0de3b4c3d82b0a8974d2f258e)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:20.191408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:20.256743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478aeac0de3b4c3d82b0a8974d2f258e', 'request_id': 'req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.262627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478aeac0de3b4c3d82b0a8974d2f258e', 'request_id': 'req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.270787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478aeac0de3b4c3d82b0a8974d2f258e', 'request_id': 'req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.282771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] The token cache is being invalidate because project 478aeac0de3b4c3d82b0a8974d2f258e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:20.283075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.283728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.285867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1169/2333] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:20 2022] DELETE /identity/v3/projects/478aeac0de3b4c3d82b0a8974d2f258e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:20.290831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:20.301211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d2c43-1995-4ed4-b93e-0fa4eb067661 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:20.360455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:20.361381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:20.361660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:20.361933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] PATH_INFO: `/v3/users/5bfce2feb8934fe69c37c490e9bec014` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:20.363177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bfce2feb8934fe69c37c490e9bec014)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:20.365226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:20.423816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bfce2feb8934fe69c37c490e9bec014'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.425228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bfce2feb8934fe69c37c490e9bec014'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.437963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bfce2feb8934fe69c37c490e9bec014'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.439039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bfce2feb8934fe69c37c490e9bec014'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.449139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e5a5b5e7-6bd6-4256-8938-df980c883abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bfce2feb8934fe69c37c490e9bec014'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:20.474800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1165/2334] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:20 2022] DELETE /identity/v3/users/5bfce2feb8934fe69c37c490e9bec014 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:21.015571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.017022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:21.041152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe1bfe-8293-45a9-bc2b-141ebfef13f2 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.083693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:21.125861 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.126827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.127416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.127591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] PATH_INFO: `/v3/projects/f8cb39ecda4c4585a5df865488513f51` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.129643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8cb39ecda4c4585a5df865488513f51)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.131602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.190124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8cb39ecda4c4585a5df865488513f51', 'request_id': 'req-532f184f-2462-4f05-b21d-242d25d7f1c5'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:21.197249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8cb39ecda4c4585a5df865488513f51', 'request_id': 'req-532f184f-2462-4f05-b21d-242d25d7f1c5'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:21.206074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8cb39ecda4c4585a5df865488513f51', 'request_id': 'req-532f184f-2462-4f05-b21d-242d25d7f1c5'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:21.218935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] The token cache is being invalidate because project f8cb39ecda4c4585a5df865488513f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:21.219224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:21.219877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-532f184f-2462-4f05-b21d-242d25d7f1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:21.222145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1170/2335] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:21 2022] DELETE /identity/v3/projects/f8cb39ecda4c4585a5df865488513f51 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:21.227638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b268b-c6ac-4610-a18d-e6e5e3b6b18e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.227926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b268b-c6ac-4610-a18d-e6e5e3b6b18e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.228207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b268b-c6ac-4610-a18d-e6e5e3b6b18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.255148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f84b268b-c6ac-4610-a18d-e6e5e3b6b18e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:21.256728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f84b268b-c6ac-4610-a18d-e6e5e3b6b18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:21.338920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1166/2336] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:21.347862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782bda9-0a26-49bd-8116-ab02e3ff991b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.348227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782bda9-0a26-49bd-8116-ab02e3ff991b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.348577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782bda9-0a26-49bd-8116-ab02e3ff991b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.367239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-4782bda9-0a26-49bd-8116-ab02e3ff991b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:21.368438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4782bda9-0a26-49bd-8116-ab02e3ff991b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:21.377145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1171/2337] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:21.381494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b268b-c6ac-4610-a18d-e6e5e3b6b18e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.388098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:21.392688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b268b-c6ac-4610-a18d-e6e5e3b6b18e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.398302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.399015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.399569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.400099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.401079 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:21.401697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.403315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.419446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1167/2338] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:06:21.420527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782bda9-0a26-49bd-8116-ab02e3ff991b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.429690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782bda9-0a26-49bd-8116-ab02e3ff991b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.434799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.435566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.435733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.436006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.436686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.438634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.466207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1172/2339] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:21 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:06:21.470080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.480457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042e827-db6e-4276-9b09-c18924f41343 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.549744 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.550329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.550607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.550884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.551571 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.553513 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.574444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1168/2340] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:21 2022] POST /identity/v3/users => generated 304 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:21.578591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.588965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72078560-4d52-4792-8f2f-ecae65fd229a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.593675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.594340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.594630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.594912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.595644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.597294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.610749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1173/2341] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:21.611938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.624334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88dc13-cbd2-4dbd-837d-94b94272be70 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.629337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d354-8067-40bb-ba3f-40c28d9cf5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.630010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d354-8067-40bb-ba3f-40c28d9cf5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.630327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d354-8067-40bb-ba3f-40c28d9cf5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.630605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d354-8067-40bb-ba3f-40c28d9cf5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.631331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8d354-8067-40bb-ba3f-40c28d9cf5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.633034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8d354-8067-40bb-ba3f-40c28d9cf5c0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.644208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1169/2342] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:21.648482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.659704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f309e-bfaa-4c95-8774-630c805fe090 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.664375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.665310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.665598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.665878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] PATH_INFO: `/v3/projects/2c0b30d345854a6e8b4ccd9f4deae039/users/604e02bcc8dc4193bb8ff8d74346e1ff/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.681282 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0b30d345854a6e8b4ccd9f4deae039, user_id=604e02bcc8dc4193bb8ff8d74346e1ff, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.683992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.698806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1174/2343] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:21 2022] PUT /identity/v3/projects/2c0b30d345854a6e8b4ccd9f4deae039/users/604e02bcc8dc4193bb8ff8d74346e1ff/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:21.708996 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf2a7-1b39-4a74-955c-1367ce0bf570 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.709284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf2a7-1b39-4a74-955c-1367ce0bf570 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.709559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf2a7-1b39-4a74-955c-1367ce0bf570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.743217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ee6cf2a7-1b39-4a74-955c-1367ce0bf570 None None] MFA Rules not processed for user `604e02bcc8dc4193bb8ff8d74346e1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:21.744403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ee6cf2a7-1b39-4a74-955c-1367ce0bf570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:21.819485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1170/2344] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:21 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:21.821237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.882617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:21.890249 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977f428-24a7-4753-bc82-8b65332d3507 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:21.896374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:21.897518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:21.897822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:21.898099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:21.898825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:21.900733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:21.931711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1175/2345] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:21 2022] POST /identity/v3/projects => generated 409 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:21.932962 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf2a7-1b39-4a74-955c-1367ce0bf570 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:21.944198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf2a7-1b39-4a74-955c-1367ce0bf570 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:22.012880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:22.013669 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.013960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.014222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.014998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:22.017145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:22.023977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:22.041626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1171/2346] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:21 2022] POST /identity/v3/users => generated 311 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:22.042951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:22.054742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31a52b-5bd5-4cd7-b41e-5fba61a2024c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:22.061807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:22.062649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.062973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.063324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.064311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:22.065890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:22.077824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1176/2347] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:22.081812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:22.093319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185b0c-639c-41f4-8044-28b2dc587e98 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:22.097993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:22.098889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.099174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.099454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] PATH_INFO: `/v3/projects/5a6a67599e77451c9d64ce4d5ed11882/users/fe26971b492a40d7859b41bcca14251d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.115404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a67599e77451c9d64ce4d5ed11882, user_id=fe26971b492a40d7859b41bcca14251d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:22.118149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:22.133483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1172/2348] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:22 2022] PUT /identity/v3/projects/5a6a67599e77451c9d64ce4d5ed11882/users/fe26971b492a40d7859b41bcca14251d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:22.138217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:22.149375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40f28c-da1d-47ed-8ec8-30238b650090 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:22.227932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97f1c8-9afb-45e9-a946-5de187012d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:22.228640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f1c8-9afb-45e9-a946-5de187012d5c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.229015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f1c8-9afb-45e9-a946-5de187012d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.229922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97f1c8-9afb-45e9-a946-5de187012d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.230658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97f1c8-9afb-45e9-a946-5de187012d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:22.232307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97f1c8-9afb-45e9-a946-5de187012d5c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:22.243466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1177/2349] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:22.247672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:22.258282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3127a68-36fe-4157-bf21-e3762d688997 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:22.263187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:22.264176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.264546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.264942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] PATH_INFO: `/v3/projects/5a6a67599e77451c9d64ce4d5ed11882/users/fe26971b492a40d7859b41bcca14251d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.266987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6a67599e77451c9d64ce4d5ed11882, user_id=fe26971b492a40d7859b41bcca14251d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:22.269803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:22.281055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1173/2350] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:22 2022] PUT /identity/v3/projects/5a6a67599e77451c9d64ce4d5ed11882/users/fe26971b492a40d7859b41bcca14251d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:22.291740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fa9c7-1da5-4fec-833f-7fd01c0e068e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.292039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fa9c7-1da5-4fec-833f-7fd01c0e068e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.292326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fa9c7-1da5-4fec-833f-7fd01c0e068e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.320853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a80fa9c7-1da5-4fec-833f-7fd01c0e068e None None] MFA Rules not processed for user `fe26971b492a40d7859b41bcca14251d`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:22.322097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a80fa9c7-1da5-4fec-833f-7fd01c0e068e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:22.393484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1178/2351] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:22.404705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:22.406745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:22.466174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902dc3c-7336-4c4d-879b-81e53afd8229 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:22.469848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f730ee-13e2-4a93-9318-3daf39d92e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:22.470500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f730ee-13e2-4a93-9318-3daf39d92e9f service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.471059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f730ee-13e2-4a93-9318-3daf39d92e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.471059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f730ee-13e2-4a93-9318-3daf39d92e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.480255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f730ee-13e2-4a93-9318-3daf39d92e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:22.481716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f730ee-13e2-4a93-9318-3daf39d92e9f service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:22.491242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1174/2352] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:22.647221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fa9c7-1da5-4fec-833f-7fd01c0e068e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:22.658647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fa9c7-1da5-4fec-833f-7fd01c0e068e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:22.662300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:22.662948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:22.663235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:22.663514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:22.717619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:22.729556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:22.731060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:22.740043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1179/2353] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:23.016605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c913ae-0d5f-4569-b7e7-1afad81258e8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:23.016920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c913ae-0d5f-4569-b7e7-1afad81258e8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:23.017200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c913ae-0d5f-4569-b7e7-1afad81258e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:23.037834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a5c913ae-0d5f-4569-b7e7-1afad81258e8 None None] MFA Rules not processed for user `b424792631ac40f48c3f3c1529d699b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:23.043305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a5c913ae-0d5f-4569-b7e7-1afad81258e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:23.098752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1175/2354] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:06:32.191132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:32.253504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5abd07-1e97-43f3-aeb8-4e29ac6bf933 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:32.258168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:32.259057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.259343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.259617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] PATH_INFO: `/v3/users/604e02bcc8dc4193bb8ff8d74346e1ff` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.261341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] RBAC: Authorizing `identity:delete_user(user_id=604e02bcc8dc4193bb8ff8d74346e1ff)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:32.263262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:32.307895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604e02bcc8dc4193bb8ff8d74346e1ff'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.308545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604e02bcc8dc4193bb8ff8d74346e1ff'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.309154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604e02bcc8dc4193bb8ff8d74346e1ff'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.315823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604e02bcc8dc4193bb8ff8d74346e1ff'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.327400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604e02bcc8dc4193bb8ff8d74346e1ff'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.340259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1180/2355] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:32 2022] DELETE /identity/v3/users/604e02bcc8dc4193bb8ff8d74346e1ff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:32.345023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.345517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.345631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.363063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:32.364263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:32.415484 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:32.425433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1176/2356] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:32.435288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:32.436731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:32.493262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63569b31-83cb-4191-baae-ff406a7de41a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:32.496867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:32.497523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.497879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.498161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.507101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:32.508548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:32.518311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1181/2357] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:32.784527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:32.785992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:32.800201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5085c-9128-45e0-aeb6-a860b5246218 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:32.805469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:32.807761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.808159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.808537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] PATH_INFO: `/v3/projects/2c0b30d345854a6e8b4ccd9f4deae039` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.810216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0b30d345854a6e8b4ccd9f4deae039)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:32.812756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:32.866890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0b30d345854a6e8b4ccd9f4deae039', 'request_id': 'req-5cf54380-de6c-4112-b971-85ecc3d97ab6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.872874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0b30d345854a6e8b4ccd9f4deae039', 'request_id': 'req-5cf54380-de6c-4112-b971-85ecc3d97ab6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.880800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0b30d345854a6e8b4ccd9f4deae039', 'request_id': 'req-5cf54380-de6c-4112-b971-85ecc3d97ab6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.894356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] The token cache is being invalidate because project 2c0b30d345854a6e8b4ccd9f4deae039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:32.894673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.895291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:32.896826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1177/2358] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:32 2022] DELETE /identity/v3/projects/2c0b30d345854a6e8b4ccd9f4deae039 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:32.901239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:32.911161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b199106-d03d-4489-808b-11caf90622ed service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:32.919493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:32.975156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:32.976015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:32.976300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:32.976583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] PATH_INFO: `/v3/users/fe26971b492a40d7859b41bcca14251d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:32.977817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe26971b492a40d7859b41bcca14251d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:32.979799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.028447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe26971b492a40d7859b41bcca14251d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.029140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe26971b492a40d7859b41bcca14251d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.029752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe26971b492a40d7859b41bcca14251d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.036859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe26971b492a40d7859b41bcca14251d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.049047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-d5f85fdf-04a3-43f1-8ad0-3d33e95269e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe26971b492a40d7859b41bcca14251d'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.066854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1182/2359] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:32 2022] DELETE /identity/v3/users/fe26971b492a40d7859b41bcca14251d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:33.333486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:33.335376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:33.351881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf54380-de6c-4112-b971-85ecc3d97ab6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:33.411420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:33.412269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.412556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.412829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] PATH_INFO: `/v3/projects/5a6a67599e77451c9d64ce4d5ed11882` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.414051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6a67599e77451c9d64ce4d5ed11882)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:33.415958 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.463942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6a67599e77451c9d64ce4d5ed11882', 'request_id': 'req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.469396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6a67599e77451c9d64ce4d5ed11882', 'request_id': 'req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.476593 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6a67599e77451c9d64ce4d5ed11882', 'request_id': 'req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.488327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] The token cache is being invalidate because project 5a6a67599e77451c9d64ce4d5ed11882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:33.488620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.489284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1fb6aabf-c454-4881-9fbf-f4d1dc4226b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:33.490816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1178/2360] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:33 2022] DELETE /identity/v3/projects/5a6a67599e77451c9d64ce4d5ed11882 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:33.495862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20fbd-f4c5-4d57-b54e-947c88cb0182 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.496335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20fbd-f4c5-4d57-b54e-947c88cb0182 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.496471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc20fbd-f4c5-4d57-b54e-947c88cb0182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.514754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-dbc20fbd-f4c5-4d57-b54e-947c88cb0182 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:33.515966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dbc20fbd-f4c5-4d57-b54e-947c88cb0182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:33.575339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1183/2361] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:33.584084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7db9c-e417-4f9f-8eda-e267b7f5db39 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.584377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7db9c-e417-4f9f-8eda-e267b7f5db39 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.584660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7db9c-e417-4f9f-8eda-e267b7f5db39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.603255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-ecd7db9c-e417-4f9f-8eda-e267b7f5db39 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:33.605159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-ecd7db9c-e417-4f9f-8eda-e267b7f5db39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:33.613165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1179/2362] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:33.617252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc20fbd-f4c5-4d57-b54e-947c88cb0182 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:33.627056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc20fbd-f4c5-4d57-b54e-947c88cb0182 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:33.631686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:33.632343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.632633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.632952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.633707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:33.634030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:33.635654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.646943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1184/2363] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:06:33.651733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7db9c-e417-4f9f-8eda-e267b7f5db39 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:33.661321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7db9c-e417-4f9f-8eda-e267b7f5db39 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:33.665962 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:33.667178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.667464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.667740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.668428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:33.670408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.673606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:33.697265 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1180/2364] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:33 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:33.701940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:33.711756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed8d94-d045-4264-aa25-1c1d07b9cb5f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:33.724347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:33.770370 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:33.771042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.771326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.771603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.772291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:33.774259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.794346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1185/2365] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:33 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:33.798650 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:33.809930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499d72b-fcc4-4395-828b-858360efbaff admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:33.813641 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:33.814301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.814582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.814859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.815581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:33.817208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.827880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1181/2366] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:33.831936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:33.842312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad2b77-eba1-46a5-8199-d5071beee108 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:33.847162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88501f-f4f4-4ec1-b5b4-bcdb49cfe014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:33.847772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88501f-f4f4-4ec1-b5b4-bcdb49cfe014 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.848057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88501f-f4f4-4ec1-b5b4-bcdb49cfe014 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.848350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88501f-f4f4-4ec1-b5b4-bcdb49cfe014 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.849082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88501f-f4f4-4ec1-b5b4-bcdb49cfe014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:33.850686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88501f-f4f4-4ec1-b5b4-bcdb49cfe014 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.861234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1186/2367] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:33.865948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:33.877173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec80bc-7db8-4982-97e1-abc31abb2af1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:33.886847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:33.887745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.888027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.888307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] PATH_INFO: `/v3/projects/528ef1ac32fb414cafb38c7b08ee2f2b/users/13263e2eaf08454d93caca9bd57eeb34/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.903782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528ef1ac32fb414cafb38c7b08ee2f2b, user_id=13263e2eaf08454d93caca9bd57eeb34, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:33.906788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:33.920177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1182/2368] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:33 2022] PUT /identity/v3/projects/528ef1ac32fb414cafb38c7b08ee2f2b/users/13263e2eaf08454d93caca9bd57eeb34/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:33.930361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02720f81-1bb8-4a06-b16d-fbf4b3038ebd None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:33.930654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02720f81-1bb8-4a06-b16d-fbf4b3038ebd None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:33.930934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02720f81-1bb8-4a06-b16d-fbf4b3038ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:33.961608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-02720f81-1bb8-4a06-b16d-fbf4b3038ebd None None] MFA Rules not processed for user `13263e2eaf08454d93caca9bd57eeb34`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:33.963410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-02720f81-1bb8-4a06-b16d-fbf4b3038ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:34.036987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1187/2369] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:34.042027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.108450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eee577-2432-46e4-8f70-139bbe84c1c6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.113196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.113863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.114148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.114453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.115145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.117081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.150206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1183/2370] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:34 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:34.152004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02720f81-1bb8-4a06-b16d-fbf4b3038ebd None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.163141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02720f81-1bb8-4a06-b16d-fbf4b3038ebd None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.223813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.224562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.224862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.225142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.225824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.227749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.247782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1188/2371] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:34 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:34.251704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.262711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632358fa-0731-4a6a-a1dd-0a527f62258d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.267369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.268023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.268310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.268589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.269473 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.271062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.281780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1184/2372] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:34.285784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.296059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d525632-5323-4305-903d-883d64b36fcf admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.300638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.301549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.301827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.302102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] PATH_INFO: `/v3/projects/d1d3910933b242eb869451c9b8a29f10/users/e1468ac829314582bb9076fcfd09f50b/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.314688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:34.317886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d3910933b242eb869451c9b8a29f10, user_id=e1468ac829314582bb9076fcfd09f50b, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.321405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.334988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1189/2373] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:34 2022] PUT /identity/v3/projects/d1d3910933b242eb869451c9b8a29f10/users/e1468ac829314582bb9076fcfd09f50b/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:34.338783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.348726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1636e-ca08-4050-89a5-1cd7b24666a1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.405993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64fd38-54a4-41da-8848-a628e8b989b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.406963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64fd38-54a4-41da-8848-a628e8b989b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.407184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64fd38-54a4-41da-8848-a628e8b989b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.407456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64fd38-54a4-41da-8848-a628e8b989b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.408176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64fd38-54a4-41da-8848-a628e8b989b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.409783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64fd38-54a4-41da-8848-a628e8b989b5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.419750 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1185/2374] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:34.423921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.434917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71639a2b-c051-4011-99d2-f7bd66aca9fc admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.439672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.440553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.440813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.441118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] PATH_INFO: `/v3/projects/d1d3910933b242eb869451c9b8a29f10/users/e1468ac829314582bb9076fcfd09f50b/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.443038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d3910933b242eb869451c9b8a29f10, user_id=e1468ac829314582bb9076fcfd09f50b, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.445713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.456152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1190/2375] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:34 2022] PUT /identity/v3/projects/d1d3910933b242eb869451c9b8a29f10/users/e1468ac829314582bb9076fcfd09f50b/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:34.467094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f169491-4a34-4860-ab3d-464cd874e62e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.467403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f169491-4a34-4860-ab3d-464cd874e62e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.467678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f169491-4a34-4860-ab3d-464cd874e62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.493683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-1f169491-4a34-4860-ab3d-464cd874e62e None None] MFA Rules not processed for user `e1468ac829314582bb9076fcfd09f50b`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:34.496139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1f169491-4a34-4860-ab3d-464cd874e62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:34.504298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:34.569156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1186/2376] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:34.580278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.582027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:34.638809 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e3a07-aa90-4861-be84-989c88561eec admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.642500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.643137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.643424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.643703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.652710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.654175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.665714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1191/2377] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:34.735541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f169491-4a34-4860-ab3d-464cd874e62e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:34.746379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f169491-4a34-4860-ab3d-464cd874e62e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:34.750039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cb495-dbbe-4352-856f-0132355f448e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:34.750700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb495-dbbe-4352-856f-0132355f448e service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:34.750974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb495-dbbe-4352-856f-0132355f448e service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:34.751252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cb495-dbbe-4352-856f-0132355f448e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:34.809578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cb495-dbbe-4352-856f-0132355f448e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:34.811034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cb495-dbbe-4352-856f-0132355f448e service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:34.820697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1187/2378] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:35.076330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:35.140243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fae66-e53d-4dea-92cf-bed9d47f26ed service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:35.144860 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:35.145723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:35.146003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:35.146274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] PATH_INFO: `/v3/users/13263e2eaf08454d93caca9bd57eeb34` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:35.147462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13263e2eaf08454d93caca9bd57eeb34)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:35.149449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:35.196424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13263e2eaf08454d93caca9bd57eeb34'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.197110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13263e2eaf08454d93caca9bd57eeb34'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.198231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13263e2eaf08454d93caca9bd57eeb34'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.205388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13263e2eaf08454d93caca9bd57eeb34'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.217611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13263e2eaf08454d93caca9bd57eeb34'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.226751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:35.233281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1192/2379] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:35 2022] DELETE /identity/v3/users/13263e2eaf08454d93caca9bd57eeb34 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:35.239237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:35.239541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:35.239818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:35.256773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:35.258024 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:35.317142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1188/2380] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:06:35.327292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:35.328724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:35.386035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a5f37-da94-4fe7-ac30-f8e253ad1cf7 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:35.389661 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:35.390293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:35.390574 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:35.390850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:35.399699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:35.401749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:35.411247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1193/2381] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:35.685844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:35.687279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:35.700330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a37f3-e73b-4759-8d9d-3cd81126e6a6 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:35.704979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:35.705874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:35.706155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:35.706428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] PATH_INFO: `/v3/projects/528ef1ac32fb414cafb38c7b08ee2f2b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:35.707636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=528ef1ac32fb414cafb38c7b08ee2f2b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:35.709687 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:35.750290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:35.758590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528ef1ac32fb414cafb38c7b08ee2f2b', 'request_id': 'req-69cc289e-1783-41c9-89d9-8a72da7bce80'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.764188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528ef1ac32fb414cafb38c7b08ee2f2b', 'request_id': 'req-69cc289e-1783-41c9-89d9-8a72da7bce80'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.771510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528ef1ac32fb414cafb38c7b08ee2f2b', 'request_id': 'req-69cc289e-1783-41c9-89d9-8a72da7bce80'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.783375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] The token cache is being invalidate because project 528ef1ac32fb414cafb38c7b08ee2f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:35.783664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.784293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.785911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1189/2382] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:35 2022] DELETE /identity/v3/projects/528ef1ac32fb414cafb38c7b08ee2f2b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:35.789869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:35.800038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82421fea-fc11-4778-a395-aa9c92d887ab service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:35.856713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:35.857551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:35.857837 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:35.858120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] PATH_INFO: `/v3/users/e1468ac829314582bb9076fcfd09f50b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:35.859304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1468ac829314582bb9076fcfd09f50b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:35.861294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:35.913504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1468ac829314582bb9076fcfd09f50b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.914320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1468ac829314582bb9076fcfd09f50b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.914937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1468ac829314582bb9076fcfd09f50b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.925122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1468ac829314582bb9076fcfd09f50b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.937795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-21b4e3bc-0192-456d-972e-866d8d191808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1468ac829314582bb9076fcfd09f50b'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:35.950716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1194/2383] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:35 2022] DELETE /identity/v3/users/e1468ac829314582bb9076fcfd09f50b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:36.259205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.262091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:36.276766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc289e-1783-41c9-89d9-8a72da7bce80 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:36.357883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:36.359613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.359932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.360219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] PATH_INFO: `/v3/projects/d1d3910933b242eb869451c9b8a29f10` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.361564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d3910933b242eb869451c9b8a29f10)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:36.363496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:36.421408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d3910933b242eb869451c9b8a29f10', 'request_id': 'req-0614b56d-b003-44f7-96b9-2b5128227c2c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:36.427786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d3910933b242eb869451c9b8a29f10', 'request_id': 'req-0614b56d-b003-44f7-96b9-2b5128227c2c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:36.435309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d3910933b242eb869451c9b8a29f10', 'request_id': 'req-0614b56d-b003-44f7-96b9-2b5128227c2c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:36.446528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] The token cache is being invalidate because project d1d3910933b242eb869451c9b8a29f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:36.446828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:36.448013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0614b56d-b003-44f7-96b9-2b5128227c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:36.449590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1190/2384] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:36 2022] DELETE /identity/v3/projects/d1d3910933b242eb869451c9b8a29f10 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:36.454671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d20b00-5613-4d25-ab7b-36e58d38bebe None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.455177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d20b00-5613-4d25-ab7b-36e58d38bebe None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.455485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d20b00-5613-4d25-ab7b-36e58d38bebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.473291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-38d20b00-5613-4d25-ab7b-36e58d38bebe None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:36.474489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38d20b00-5613-4d25-ab7b-36e58d38bebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:36.490902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:36.537450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1195/2385] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:06:36.545438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484d43-dcb3-415e-9453-1f95a5046f8b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.545734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484d43-dcb3-415e-9453-1f95a5046f8b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.546025 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9484d43-dcb3-415e-9453-1f95a5046f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.564591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b9484d43-dcb3-415e-9453-1f95a5046f8b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:36.565842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b9484d43-dcb3-415e-9453-1f95a5046f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:36.574305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1191/2386] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:36.578413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d20b00-5613-4d25-ab7b-36e58d38bebe None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.588121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d20b00-5613-4d25-ab7b-36e58d38bebe None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:36.592992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:36.593651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.593937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.594218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.594942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:36.595255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:36.596931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:36.609073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1196/2387] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:06:36.613467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9484d43-dcb3-415e-9453-1f95a5046f8b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.623183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9484d43-dcb3-415e-9453-1f95a5046f8b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:36.627841 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:36.628493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.628795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.629087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.629779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:36.631685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:36.658642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1192/2388] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:36 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:36.662793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.673266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80294e3-08c5-4836-ad7a-6b6c64f90c1f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:36.729390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:36.730307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.730597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.730880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.731606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:36.733570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:36.753958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1197/2389] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:36 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:06:36.758004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.768145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ba1d-5885-4994-a83d-99d3ded39b54 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:36.772868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:36.773521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.773807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.774082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.774801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:36.776383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:36.786887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1193/2390] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:36.790579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.801050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6bb8-9b3b-4ebe-b323-ee3d8f31dc98 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:36.805685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475fee7-05fe-43f4-83ba-e8f82fa0b3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:36.806443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fee7-05fe-43f4-83ba-e8f82fa0b3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.807251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fee7-05fe-43f4-83ba-e8f82fa0b3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.807539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fee7-05fe-43f4-83ba-e8f82fa0b3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.808272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fee7-05fe-43f4-83ba-e8f82fa0b3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:36.809940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fee7-05fe-43f4-83ba-e8f82fa0b3b3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:36.821694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1198/2391] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:36.825835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.835719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff574-b26d-45e8-9d24-ee496aae6666 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:36.840406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:36.841482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.841766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.842046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] PATH_INFO: `/v3/projects/83da07ebe0d7454a870e41b7f9bbf18b/users/f335f4d091e74d99b6cb2971b5f96d0f/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.857169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83da07ebe0d7454a870e41b7f9bbf18b, user_id=f335f4d091e74d99b6cb2971b5f96d0f, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:36.859901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:36.862026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:36.875302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1194/2392] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:36 2022] PUT /identity/v3/projects/83da07ebe0d7454a870e41b7f9bbf18b/users/f335f4d091e74d99b6cb2971b5f96d0f/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:36.885599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619447d7-06c2-45b4-89f3-fd39fa0991b0 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:36.885925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619447d7-06c2-45b4-89f3-fd39fa0991b0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:36.886204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619447d7-06c2-45b4-89f3-fd39fa0991b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:36.912376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-619447d7-06c2-45b4-89f3-fd39fa0991b0 None None] MFA Rules not processed for user `f335f4d091e74d99b6cb2971b5f96d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:36.913658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-619447d7-06c2-45b4-89f3-fd39fa0991b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:36.977115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1199/2393] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:36.988638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:36.990356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:37.047114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc30a-9d80-4011-92da-da8191f42987 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:37.050754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84ac94-62ba-4e26-943c-106460731d67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:37.051392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84ac94-62ba-4e26-943c-106460731d67 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.051684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84ac94-62ba-4e26-943c-106460731d67 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.051959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84ac94-62ba-4e26-943c-106460731d67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.060858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84ac94-62ba-4e26-943c-106460731d67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:37.062280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84ac94-62ba-4e26-943c-106460731d67 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:37.071814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1195/2394] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:37.093685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619447d7-06c2-45b4-89f3-fd39fa0991b0 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:37.113262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:37.158734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619447d7-06c2-45b4-89f3-fd39fa0991b0 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:37.163336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:37.164206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.164495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.164774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] PATH_INFO: `/v3/users/f335f4d091e74d99b6cb2971b5f96d0f` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.166630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f335f4d091e74d99b6cb2971b5f96d0f)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:37.168572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:37.214430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f335f4d091e74d99b6cb2971b5f96d0f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.215083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f335f4d091e74d99b6cb2971b5f96d0f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.215691 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f335f4d091e74d99b6cb2971b5f96d0f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.222743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f335f4d091e74d99b6cb2971b5f96d0f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.234669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f335f4d091e74d99b6cb2971b5f96d0f'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.248138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1200/2395] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:37 2022] DELETE /identity/v3/users/f335f4d091e74d99b6cb2971b5f96d0f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:37.252657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.252992 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.253273 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.272548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:37.273956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:37.335423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1196/2396] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:37.347100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:37.348507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:37.405807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e8185-82a1-459b-90bb-9f79e1729b9c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:37.409769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42f947-bcc4-4fc8-9f63-95b2cbf179ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:37.410418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42f947-bcc4-4fc8-9f63-95b2cbf179ff service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.410803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42f947-bcc4-4fc8-9f63-95b2cbf179ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.411030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42f947-bcc4-4fc8-9f63-95b2cbf179ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.421482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42f947-bcc4-4fc8-9f63-95b2cbf179ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:37.422978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42f947-bcc4-4fc8-9f63-95b2cbf179ff service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:37.432658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1201/2397] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:37.699019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:37.700471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:37.712801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c0a6c-8a38-429e-b8af-9a7e62cf23d8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:37.718067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:37.718926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.719214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.719498 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] PATH_INFO: `/v3/projects/83da07ebe0d7454a870e41b7f9bbf18b` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.720739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83da07ebe0d7454a870e41b7f9bbf18b)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:37.722682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:37.768963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83da07ebe0d7454a870e41b7f9bbf18b', 'request_id': 'req-f7d1f350-a7cf-4aff-8533-3db089891544'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.774664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83da07ebe0d7454a870e41b7f9bbf18b', 'request_id': 'req-f7d1f350-a7cf-4aff-8533-3db089891544'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.782556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83da07ebe0d7454a870e41b7f9bbf18b', 'request_id': 'req-f7d1f350-a7cf-4aff-8533-3db089891544'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.793540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] The token cache is being invalidate because project 83da07ebe0d7454a870e41b7f9bbf18b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:37.793941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.794573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f7d1f350-a7cf-4aff-8533-3db089891544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:37.796101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1197/2398] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:37 2022] DELETE /identity/v3/projects/83da07ebe0d7454a870e41b7f9bbf18b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:37.800979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11714c-08c7-48be-bb0b-d783b7e28fa8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.801473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11714c-08c7-48be-bb0b-d783b7e28fa8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.801581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11714c-08c7-48be-bb0b-d783b7e28fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.820011 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-4f11714c-08c7-48be-bb0b-d783b7e28fa8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:37.821287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4f11714c-08c7-48be-bb0b-d783b7e28fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:37.855479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:37.883038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1202/2399] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:37.891646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3ce8-f06c-482b-a5d4-af84b13f94ed None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.891936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3ce8-f06c-482b-a5d4-af84b13f94ed None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.892213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3ce8-f06c-482b-a5d4-af84b13f94ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.911043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-aebd3ce8-f06c-482b-a5d4-af84b13f94ed None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:37.912241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-aebd3ce8-f06c-482b-a5d4-af84b13f94ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:37.920794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1198/2400] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:37.924886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11714c-08c7-48be-bb0b-d783b7e28fa8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:37.934833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11714c-08c7-48be-bb0b-d783b7e28fa8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:37.939617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:37.940367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.940645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.940955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.941734 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:37.941995 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:37.943679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:37.955509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1203/2401] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:06:37.959655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3ce8-f06c-482b-a5d4-af84b13f94ed None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:37.969367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3ce8-f06c-482b-a5d4-af84b13f94ed None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:37.974048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:37.974723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:37.975003 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:37.975272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:37.975957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:37.977876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:37.988347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:38.006840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1199/2402] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:37 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:38.011227 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.021266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b966b0-3264-4475-9847-35b08fd13b5b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.086325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.087162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.087461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.087722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.088993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.090970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.110949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1204/2403] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:38 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:06:38.115012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.125118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869af74c-08af-431b-9749-9645b1a588c8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.129844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.130490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.130770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.131043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.131758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.133382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.143604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1200/2404] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:38.147633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.158284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc3ad4-87ea-45ff-bb04-8ddfe2f1d9b5 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.162889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d52e2c-c172-4b32-b96b-aa0c9da8429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.163541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d52e2c-c172-4b32-b96b-aa0c9da8429f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.163829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d52e2c-c172-4b32-b96b-aa0c9da8429f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.164109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d52e2c-c172-4b32-b96b-aa0c9da8429f admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.164857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d52e2c-c172-4b32-b96b-aa0c9da8429f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.167086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d52e2c-c172-4b32-b96b-aa0c9da8429f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.176841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1205/2405] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:38.180653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.190589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cde055-dfa6-4891-8f51-848e4e089da7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.195404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.196258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.196543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.196820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] PATH_INFO: `/v3/projects/3afa8600465b4f6fbe0a0a97a2e21be6/users/8f2829edc5c4456b96b05c4fd4d75a22/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.213481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afa8600465b4f6fbe0a0a97a2e21be6, user_id=8f2829edc5c4456b96b05c4fd4d75a22, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.216152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.228793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1201/2406] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:38 2022] PUT /identity/v3/projects/3afa8600465b4f6fbe0a0a97a2e21be6/users/8f2829edc5c4456b96b05c4fd4d75a22/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:38.239292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55e2be-1d60-4ba0-ba23-a50cd41c8de1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.239581 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55e2be-1d60-4ba0-ba23-a50cd41c8de1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.239857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55e2be-1d60-4ba0-ba23-a50cd41c8de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.266492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fc55e2be-1d60-4ba0-ba23-a50cd41c8de1 None None] MFA Rules not processed for user `8f2829edc5c4456b96b05c4fd4d75a22`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:38.267708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fc55e2be-1d60-4ba0-ba23-a50cd41c8de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:38.332010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1206/2407] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:38.336512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.397936 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30788460-efcc-4346-9256-78dbea5acd1b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.403444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.404229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.404510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.404782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.405504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.407657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.431987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1202/2408] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:38 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:38.436082 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55e2be-1d60-4ba0-ba23-a50cd41c8de1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.446932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55e2be-1d60-4ba0-ba23-a50cd41c8de1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.479346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:38.508129 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.509410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.510219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.510657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.511783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.514068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.542179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1207/2409] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:38 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:06:38.543395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.553701 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d5aa4-cde0-48e4-b857-6eab7537d5b1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.558925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.559578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.559865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.560156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.560898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.562533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.573675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1203/2410] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:38.577968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.588451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45656c92-82f6-4271-bf20-8df757a7b474 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.593206 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.594092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.594386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.594668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] PATH_INFO: `/v3/projects/811d4553b34144d680cdee21ad97748b/users/a1139c15f9834d4bb2f406b618fb8f20/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.611022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] RBAC: Authorizing `identity:create_grant(project_id=811d4553b34144d680cdee21ad97748b, user_id=a1139c15f9834d4bb2f406b618fb8f20, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.613773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.628020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1208/2411] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:38 2022] PUT /identity/v3/projects/811d4553b34144d680cdee21ad97748b/users/a1139c15f9834d4bb2f406b618fb8f20/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:38.632166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.643066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd8689-f12b-4590-b3ef-da3cdbe03638 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.702832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcddd2-83a0-4ffc-b150-fd092f93038a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.703418 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcddd2-83a0-4ffc-b150-fd092f93038a admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.703717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcddd2-83a0-4ffc-b150-fd092f93038a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.703970 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcddd2-83a0-4ffc-b150-fd092f93038a admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.704693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcddd2-83a0-4ffc-b150-fd092f93038a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.706310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcddd2-83a0-4ffc-b150-fd092f93038a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.717174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1204/2412] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:38.721306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.732524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f16ee-f02d-4b4a-a6d7-cd64cbf2000f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.737368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.738280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.738564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.738843 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] PATH_INFO: `/v3/projects/811d4553b34144d680cdee21ad97748b/users/a1139c15f9834d4bb2f406b618fb8f20/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.741119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811d4553b34144d680cdee21ad97748b, user_id=a1139c15f9834d4bb2f406b618fb8f20, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:38.743893 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:38.754565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1209/2413] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:38 2022] PUT /identity/v3/projects/811d4553b34144d680cdee21ad97748b/users/a1139c15f9834d4bb2f406b618fb8f20/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:38.764672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f7ec3-04dd-4c46-8856-859b3fbdf28f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.765017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f7ec3-04dd-4c46-8856-859b3fbdf28f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.765300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f7ec3-04dd-4c46-8856-859b3fbdf28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:38.793361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4b4f7ec3-04dd-4c46-8856-859b3fbdf28f None None] MFA Rules not processed for user `a1139c15f9834d4bb2f406b618fb8f20`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:38.794607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4b4f7ec3-04dd-4c46-8856-859b3fbdf28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:38.808947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:38.874143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1205/2414] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:38.885810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:38.887547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:38.948902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef3dfd-0947-4e20-81fd-7f2227ca38d1 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:38.952594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362248cf-fb7f-49b5-bb9f-a6b0324ec0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:38.953352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362248cf-fb7f-49b5-bb9f-a6b0324ec0ff service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:38.953589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362248cf-fb7f-49b5-bb9f-a6b0324ec0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:38.953871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362248cf-fb7f-49b5-bb9f-a6b0324ec0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:39.016032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362248cf-fb7f-49b5-bb9f-a6b0324ec0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:39.017553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362248cf-fb7f-49b5-bb9f-a6b0324ec0ff service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:39.028435 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1210/2415] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:41.587834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f7ec3-04dd-4c46-8856-859b3fbdf28f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:41.649159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f7ec3-04dd-4c46-8856-859b3fbdf28f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:41.653759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:41.654627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:41.654910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:41.655184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] PATH_INFO: `/v3/users/8f2829edc5c4456b96b05c4fd4d75a22` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:41.656356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2829edc5c4456b96b05c4fd4d75a22)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:41.658560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:41.707913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2829edc5c4456b96b05c4fd4d75a22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:41.708581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2829edc5c4456b96b05c4fd4d75a22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:41.715264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2829edc5c4456b96b05c4fd4d75a22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:41.715882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2829edc5c4456b96b05c4fd4d75a22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:41.722811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2829edc5c4456b96b05c4fd4d75a22'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:41.739951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1206/2416] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:41 2022] DELETE /identity/v3/users/8f2829edc5c4456b96b05c4fd4d75a22 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:41.750019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:41.750565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:41.750565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:41.768410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:41.769655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:41.807509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:41.829646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1211/2417] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:41.839667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:41.841105 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:41.876991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:41.902463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb540c-df93-40ca-840f-b4a3423ef979 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:41.906123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:41.907022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:41.907307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:41.907581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:41.917054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:41.918547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:41.927879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1207/2418] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:42.194970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:42.196532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:42.209077 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb70d0-3e25-4fa7-b56a-fee457b55886 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:42.213935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:42.214789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:42.215073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:42.215357 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] PATH_INFO: `/v3/projects/3afa8600465b4f6fbe0a0a97a2e21be6` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:42.216661 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3afa8600465b4f6fbe0a0a97a2e21be6)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:42.218524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:42.266862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3afa8600465b4f6fbe0a0a97a2e21be6', 'request_id': 'req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.272250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3afa8600465b4f6fbe0a0a97a2e21be6', 'request_id': 'req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.281051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3afa8600465b4f6fbe0a0a97a2e21be6', 'request_id': 'req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.292017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] The token cache is being invalidate because project 3afa8600465b4f6fbe0a0a97a2e21be6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:42.292348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.293040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.294605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1212/2419] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:42 2022] DELETE /identity/v3/projects/3afa8600465b4f6fbe0a0a97a2e21be6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:42.298563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:42.308129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd8803-a7b3-46e9-8c6c-0359c43a8336 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:42.363734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:42.364528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:42.364810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:42.365114 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] PATH_INFO: `/v3/users/a1139c15f9834d4bb2f406b618fb8f20` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:42.366321 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1139c15f9834d4bb2f406b618fb8f20)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:42.368231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:42.413760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1139c15f9834d4bb2f406b618fb8f20'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.414396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1139c15f9834d4bb2f406b618fb8f20'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.421767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1139c15f9834d4bb2f406b618fb8f20'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.422384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1139c15f9834d4bb2f406b618fb8f20'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.429375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d0331d1f-af7d-42cd-86c2-a80b4475a2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1139c15f9834d4bb2f406b618fb8f20'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.446427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1208/2420] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:42 2022] DELETE /identity/v3/users/a1139c15f9834d4bb2f406b618fb8f20 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:42.722561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:42.724242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:42.738291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5061b0b-2c52-4dbb-a968-d99f10cd0a1a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:42.796196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:42.797089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:42.797377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:42.797662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] PATH_INFO: `/v3/projects/811d4553b34144d680cdee21ad97748b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:42.798840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811d4553b34144d680cdee21ad97748b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:42.800772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:42.847144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811d4553b34144d680cdee21ad97748b', 'request_id': 'req-e513b28a-578a-40e6-8b49-6d8f7d9b1274'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.852597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811d4553b34144d680cdee21ad97748b', 'request_id': 'req-e513b28a-578a-40e6-8b49-6d8f7d9b1274'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.859633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811d4553b34144d680cdee21ad97748b', 'request_id': 'req-e513b28a-578a-40e6-8b49-6d8f7d9b1274'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.861554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:42.872332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] The token cache is being invalidate because project 811d4553b34144d680cdee21ad97748b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:42.872626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.873302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e513b28a-578a-40e6-8b49-6d8f7d9b1274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:42.875171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1213/2421] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:42 2022] DELETE /identity/v3/projects/811d4553b34144d680cdee21ad97748b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:42.879931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589613f8-475d-418f-be7e-36217c486e25 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:42.880422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589613f8-475d-418f-be7e-36217c486e25 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:42.880544 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589613f8-475d-418f-be7e-36217c486e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:42.898675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-589613f8-475d-418f-be7e-36217c486e25 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:42.899818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-589613f8-475d-418f-be7e-36217c486e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:42.946967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:42.961831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1209/2422] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:43.022730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d397b-e0b9-4fb3-9da8-4bf3807be6ae None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.023020 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d397b-e0b9-4fb3-9da8-4bf3807be6ae None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.023306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d397b-e0b9-4fb3-9da8-4bf3807be6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.042288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b43d397b-e0b9-4fb3-9da8-4bf3807be6ae None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:43.043495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b43d397b-e0b9-4fb3-9da8-4bf3807be6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:43.051405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1214/2423] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:43.055615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589613f8-475d-418f-be7e-36217c486e25 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.065913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589613f8-475d-418f-be7e-36217c486e25 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.070567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.071222 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.071506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.071784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.073314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:43.073625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.075251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.087190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1210/2424] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:06:43.091443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d397b-e0b9-4fb3-9da8-4bf3807be6ae None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.100595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d397b-e0b9-4fb3-9da8-4bf3807be6ae None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.105374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.106021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.106330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.106610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.107304 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.109272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.135839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1215/2425] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:43.140164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.150360 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fafca-72eb-45cb-925f-1e9d55106789 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.205342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.206001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.206283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.206554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.207232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.209173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.228205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1211/2426] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/users => generated 292 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:43.232284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.242565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244a899-8bad-445e-90d3-434a42c059a8 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.248140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.248808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.249130 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.249412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.250172 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.251788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.262377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1216/2427] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:43.266410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.277020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60adec-c439-4dab-8f5c-f9032ace4afc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.281931 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf88225-41dd-4fb7-9f9c-90d9c3a5d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.282582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf88225-41dd-4fb7-9f9c-90d9c3a5d18c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.282864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf88225-41dd-4fb7-9f9c-90d9c3a5d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.283138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf88225-41dd-4fb7-9f9c-90d9c3a5d18c admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.283860 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf88225-41dd-4fb7-9f9c-90d9c3a5d18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.285479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf88225-41dd-4fb7-9f9c-90d9c3a5d18c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.295482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1212/2428] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:43.299682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.310194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9182b-753d-44f0-9803-9d55663be4c3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.314828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.315698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.315981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.316260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] PATH_INFO: `/v3/projects/ab0d7241e4544de693a17275b7f12466/users/5a47b65073474b98b30831d76a69c79e/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.331279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0d7241e4544de693a17275b7f12466, user_id=5a47b65073474b98b30831d76a69c79e, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.334027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.347192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1217/2429] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:43 2022] PUT /identity/v3/projects/ab0d7241e4544de693a17275b7f12466/users/5a47b65073474b98b30831d76a69c79e/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:43.357085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b7d3c-c54b-4570-abde-09a096f928a3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.357388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b7d3c-c54b-4570-abde-09a096f928a3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.357667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b7d3c-c54b-4570-abde-09a096f928a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.383718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-8c2b7d3c-c54b-4570-abde-09a096f928a3 None None] MFA Rules not processed for user `5a47b65073474b98b30831d76a69c79e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:43.385590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8c2b7d3c-c54b-4570-abde-09a096f928a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:43.447802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1213/2430] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:43.452347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.513324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eee4dc-29cd-47d5-8a8e-16fee44888f1 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.518017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.518674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.518963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.519247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.519957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.521968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.547558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1218/2431] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:06:43.551720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b7d3c-c54b-4570-abde-09a096f928a3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.563085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b7d3c-c54b-4570-abde-09a096f928a3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.607335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:43.620999 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.621668 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.621979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.622256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.622942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.624877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.644942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1214/2432] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:06:43.649158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.659774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e78651-137b-4907-81bd-309a3da688b5 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.664362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.665055 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.665342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.665620 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.666832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.668427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.680118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1219/2433] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:43.684453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.694420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf51479-506b-465c-8904-4eec3bcc7c62 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.699658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c8c05-ce20-4a55-8ee0-1c7aaa098c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.700249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c8c05-ce20-4a55-8ee0-1c7aaa098c7e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.700531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c8c05-ce20-4a55-8ee0-1c7aaa098c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.700806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c8c05-ce20-4a55-8ee0-1c7aaa098c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.701564 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c8c05-ce20-4a55-8ee0-1c7aaa098c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.703148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c8c05-ce20-4a55-8ee0-1c7aaa098c7e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.713557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1215/2434] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:43.718181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.727930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39414ac-f445-4995-a484-3f452d4571ef admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.732430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.733335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.733619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.733890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] PATH_INFO: `/v3/projects/ebbc0e09731e44b3b7a3e41c1de5095d/users/965b9fa822784a5ea1a27d9ee4b6e9ec/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.748031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbc0e09731e44b3b7a3e41c1de5095d, user_id=965b9fa822784a5ea1a27d9ee4b6e9ec, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.750765 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.763932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1220/2435] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:43 2022] PUT /identity/v3/projects/ebbc0e09731e44b3b7a3e41c1de5095d/users/965b9fa822784a5ea1a27d9ee4b6e9ec/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:43.774314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3cb1-f771-4140-9650-347ca5272e54 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.774661 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3cb1-f771-4140-9650-347ca5272e54 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.774974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3cb1-f771-4140-9650-347ca5272e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.801904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-62cd3cb1-f771-4140-9650-347ca5272e54 None None] MFA Rules not processed for user `965b9fa822784a5ea1a27d9ee4b6e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:43.803113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-62cd3cb1-f771-4140-9650-347ca5272e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:43.866571 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1216/2436] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:43.871647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.916192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:43.934717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64101-ffbb-41d9-95e2-d4a0b60a7462 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:43.939447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:43.940103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:43.940397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:43.940673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:43.941405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:43.943330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:43.968013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1221/2437] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:43.972154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd3cb1-f771-4140-9650-347ca5272e54 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:43.982873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd3cb1-f771-4140-9650-347ca5272e54 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:44.038623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:44.039287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.039572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.039854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.040550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:44.043092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:44.062997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1217/2438] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:43 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:44.067126 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:44.077393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f8d11-3116-4dca-ba56-e61623f83f8f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:44.082026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:44.082683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.082973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.083254 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.083976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:44.085636 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:44.096482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1222/2439] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:44.100414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:44.111006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42f4ab-074e-45ba-bbc7-46e8f8ce097c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:44.115587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:44.116452 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.116741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.117039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] PATH_INFO: `/v3/projects/ae422a58b2de4ccf833b5284e42d4f11/users/7f7a3399f2fe4ce1b6d51ca5416b7058/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.131431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae422a58b2de4ccf833b5284e42d4f11, user_id=7f7a3399f2fe4ce1b6d51ca5416b7058, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:44.134118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:44.147355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1218/2440] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:44 2022] PUT /identity/v3/projects/ae422a58b2de4ccf833b5284e42d4f11/users/7f7a3399f2fe4ce1b6d51ca5416b7058/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:44.151787 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:44.162101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf81a3-0d6f-478d-8405-7551549c5327 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:44.219364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb4339-8024-4889-aca9-ffb9d6cb38fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:44.219960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb4339-8024-4889-aca9-ffb9d6cb38fb admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.220251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb4339-8024-4889-aca9-ffb9d6cb38fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.220534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb4339-8024-4889-aca9-ffb9d6cb38fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.221300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb4339-8024-4889-aca9-ffb9d6cb38fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:44.222908 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb4339-8024-4889-aca9-ffb9d6cb38fb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:44.233074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1223/2441] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:44.237232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:44.248189 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a22fbc-55eb-47fb-aaf9-629ce55017d6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:44.253021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:44.253932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.254217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.254497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] PATH_INFO: `/v3/projects/ae422a58b2de4ccf833b5284e42d4f11/users/7f7a3399f2fe4ce1b6d51ca5416b7058/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.256511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae422a58b2de4ccf833b5284e42d4f11, user_id=7f7a3399f2fe4ce1b6d51ca5416b7058, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:44.259242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:44.271018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1219/2442] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:44 2022] PUT /identity/v3/projects/ae422a58b2de4ccf833b5284e42d4f11/users/7f7a3399f2fe4ce1b6d51ca5416b7058/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:44.282682 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.282977 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.283260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.310729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] MFA Rules not processed for user `7f7a3399f2fe4ce1b6d51ca5416b7058`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:44.311940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:44.382743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1224/2443] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:44.393887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:44.395622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:44.406844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:44.456400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0ac8-a5cc-4f5c-8d5c-700ab62bb5b6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:44.460035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:44.460660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:44.460983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:44.461258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:44.516270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:44.517657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:44.529021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1220/2444] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:46.002500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:46.004076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:46.016801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299c8cd-e138-4c9a-836f-05b05e36fc46 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:46.020408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:46.021095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:46.021387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:46.021666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:46.057033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:46.082026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:46.083453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:46.092513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1225/2445] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:46.609595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:46.618333 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92784a83-622e-4a7f-aec4-d835ca7a1b20 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:46.622486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:46.623184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:46.623408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:46.623659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:46.633122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:46.634782 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:46.644939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1221/2446] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:51.510617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:51.573451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f4493-468d-4457-860d-8798d77745d6 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:51.578131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:51.579136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:51.579370 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:51.579707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] PATH_INFO: `/v3/users/5a47b65073474b98b30831d76a69c79e` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:51.581035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a47b65073474b98b30831d76a69c79e)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:51.581547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:51.583066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:51.590586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6188c6-a693-4ee0-955b-50c0ca878a54 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:51.594299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:51.594943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:51.595212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:51.595485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:51.597268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:51.636257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a47b65073474b98b30831d76a69c79e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:51.636975 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a47b65073474b98b30831d76a69c79e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:51.637845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a47b65073474b98b30831d76a69c79e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:51.646901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a47b65073474b98b30831d76a69c79e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:51.659362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-854d7b12-3042-4f84-883c-9779eb41b2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a47b65073474b98b30831d76a69c79e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:51.664221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:51.666239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:51.673123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1226/2447] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:51 2022] DELETE /identity/v3/users/5a47b65073474b98b30831d76a69c79e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:51.676057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1222/2448] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:51.677451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:51.677933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:51.678219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:51.695877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:51.697651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:51.754831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:51.771262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1227/2449] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:06:51.781427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:51.783090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:51.849399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=100 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:51.854820 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54049bd1-b762-4b54-a741-439334934579 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:51.906264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:51.907290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:51.907579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:51.907858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:51.917534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:51.918963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:51.928286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1223/2450] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:52.194849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:52.196330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:52.208474 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46c90b-8009-45ff-8869-3f695aef3ca1 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:52.213814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:52.214676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:52.214960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:52.215239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] PATH_INFO: `/v3/projects/ab0d7241e4544de693a17275b7f12466` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:52.216465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0d7241e4544de693a17275b7f12466)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:52.218443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:52.264684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0d7241e4544de693a17275b7f12466', 'request_id': 'req-5ea53a46-71bc-40a1-9114-987257fa4db2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.270158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0d7241e4544de693a17275b7f12466', 'request_id': 'req-5ea53a46-71bc-40a1-9114-987257fa4db2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.277804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0d7241e4544de693a17275b7f12466', 'request_id': 'req-5ea53a46-71bc-40a1-9114-987257fa4db2'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.289181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] The token cache is being invalidate because project ab0d7241e4544de693a17275b7f12466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:52.289470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.290099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.292324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1228/2451] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:52 2022] DELETE /identity/v3/projects/ab0d7241e4544de693a17275b7f12466 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:52.296240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:52.305834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de0d85-338f-4363-9524-5e9e541e8c57 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:52.368203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:52.369628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:52.369915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:52.370192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] PATH_INFO: `/v3/users/965b9fa822784a5ea1a27d9ee4b6e9ec` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:52.371429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=965b9fa822784a5ea1a27d9ee4b6e9ec)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:52.373399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:52.420583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '965b9fa822784a5ea1a27d9ee4b6e9ec'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.421327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '965b9fa822784a5ea1a27d9ee4b6e9ec'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.428187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '965b9fa822784a5ea1a27d9ee4b6e9ec'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.428814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '965b9fa822784a5ea1a27d9ee4b6e9ec'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.435957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '965b9fa822784a5ea1a27d9ee4b6e9ec'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.453657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1224/2452] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:52 2022] DELETE /identity/v3/users/965b9fa822784a5ea1a27d9ee4b6e9ec => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:52.729405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:52.730886 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:52.744146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea53a46-71bc-40a1-9114-987257fa4db2 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:52.802228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:52.803093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:52.803383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:52.803667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] PATH_INFO: `/v3/projects/ebbc0e09731e44b3b7a3e41c1de5095d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:52.804916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebbc0e09731e44b3b7a3e41c1de5095d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:52.807580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:52.853362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebbc0e09731e44b3b7a3e41c1de5095d', 'request_id': 'req-2f606fed-7c50-4a0e-8bf9-000f5878b695'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.858740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebbc0e09731e44b3b7a3e41c1de5095d', 'request_id': 'req-2f606fed-7c50-4a0e-8bf9-000f5878b695'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.870448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebbc0e09731e44b3b7a3e41c1de5095d', 'request_id': 'req-2f606fed-7c50-4a0e-8bf9-000f5878b695'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.889395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] The token cache is being invalidate because project ebbc0e09731e44b3b7a3e41c1de5095d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:52.889694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.890376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:52.892128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1229/2453] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:52 2022] DELETE /identity/v3/projects/ebbc0e09731e44b3b7a3e41c1de5095d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:52.896010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:52.897467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:52.910715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171dc75-1ace-4788-ba61-355f2a67dd3f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:52.935342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:52.971521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:52.972403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:52.972690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:52.973004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] PATH_INFO: `/v3/users/7f7a3399f2fe4ce1b6d51ca5416b7058` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:52.974213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7a3399f2fe4ce1b6d51ca5416b7058)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:52.976120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.022738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7a3399f2fe4ce1b6d51ca5416b7058'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.023385 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7a3399f2fe4ce1b6d51ca5416b7058'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.030293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7a3399f2fe4ce1b6d51ca5416b7058'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.030928 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7a3399f2fe4ce1b6d51ca5416b7058'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.038664 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fe74e565-66e3-4a49-b045-dc89588c40c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7a3399f2fe4ce1b6d51ca5416b7058'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.056854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1225/2454] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:06:52 2022] DELETE /identity/v3/users/7f7a3399f2fe4ce1b6d51ca5416b7058 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:53.332359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:53.334112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:53.348024 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f606fed-7c50-4a0e-8bf9-000f5878b695 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:53.353350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:53.411536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:53.412396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.412694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.412987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] PATH_INFO: `/v3/projects/ae422a58b2de4ccf833b5284e42d4f11` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.414156 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae422a58b2de4ccf833b5284e42d4f11)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:53.416135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.461087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae422a58b2de4ccf833b5284e42d4f11', 'request_id': 'req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.467089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae422a58b2de4ccf833b5284e42d4f11', 'request_id': 'req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.474208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae422a58b2de4ccf833b5284e42d4f11', 'request_id': 'req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.485113 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] The token cache is being invalidate because project ae422a58b2de4ccf833b5284e42d4f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:06:53.485408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.486041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1cbdb90f-6085-4fc8-bde5-9b0e881f8b76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:06:53.487651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1230/2455] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:06:53 2022] DELETE /identity/v3/projects/ae422a58b2de4ccf833b5284e42d4f11 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:53.492524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f4eda-c2f9-48a6-8e97-6830ebcfdc91 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.492814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f4eda-c2f9-48a6-8e97-6830ebcfdc91 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.493123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f4eda-c2f9-48a6-8e97-6830ebcfdc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.510856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-872f4eda-c2f9-48a6-8e97-6830ebcfdc91 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:53.512056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-872f4eda-c2f9-48a6-8e97-6830ebcfdc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:53.573654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1226/2456] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:53.581606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5f74d-f798-4209-aeb3-0c70c615e313 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.581836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5f74d-f798-4209-aeb3-0c70c615e313 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.582117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5f74d-f798-4209-aeb3-0c70c615e313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.599903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-62a5f74d-f798-4209-aeb3-0c70c615e313 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:53.601255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-62a5f74d-f798-4209-aeb3-0c70c615e313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:53.609147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1231/2457] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:53.613277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f4eda-c2f9-48a6-8e97-6830ebcfdc91 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:53.623414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f4eda-c2f9-48a6-8e97-6830ebcfdc91 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:53.628247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:53.628944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.629236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.629518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.630244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:06:53.630557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:53.632150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.644409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1227/2458] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:06:53.648488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5f74d-f798-4209-aeb3-0c70c615e313 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:53.658086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5f74d-f798-4209-aeb3-0c70c615e313 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:53.662743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:53.663394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.663680 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.663959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.664644 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:53.666625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.692118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1232/2459] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:53 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:53.695861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:53.706394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b649dbb-910e-4a0e-b6ec-f97cd41d541e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:53.735666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:53.765658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:53.766327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.766658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.766934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.767628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:53.769607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.789848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1228/2460] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:53 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:53.794060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:53.804347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ffc82-97a1-4064-893e-4554ae7c4e11 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:53.809538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:53.810195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.810685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.810943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.811684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:53.813334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.823899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1233/2461] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:53.827849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:53.838316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c39fee-5388-410c-b72a-2752ec97898d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:53.842976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdfbf0-90c5-4665-916f-16689b2a7788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:53.843623 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdfbf0-90c5-4665-916f-16689b2a7788 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.843910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdfbf0-90c5-4665-916f-16689b2a7788 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.844185 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdfbf0-90c5-4665-916f-16689b2a7788 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.844939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdfbf0-90c5-4665-916f-16689b2a7788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:53.846515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdfbf0-90c5-4665-916f-16689b2a7788 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.856811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1229/2462] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:53.861107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:53.871176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d205ef-f3b6-4e12-8b4f-859f13d47330 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:53.875828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:53.876735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.877060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.877349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] PATH_INFO: `/v3/projects/3716e192b1194365acb5595c8f92a862/users/0e6fbab71db54b698572bb8a7d033063/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.893149 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3716e192b1194365acb5595c8f92a862, user_id=0e6fbab71db54b698572bb8a7d033063, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:53.895840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:53.909710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1234/2463] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:53 2022] PUT /identity/v3/projects/3716e192b1194365acb5595c8f92a862/users/0e6fbab71db54b698572bb8a7d033063/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:06:53.914846 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42780a2a-1d43-4dd4-9fa2-2025ae91475e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:53.915317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42780a2a-1d43-4dd4-9fa2-2025ae91475e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:53.916128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42780a2a-1d43-4dd4-9fa2-2025ae91475e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:53.933879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-42780a2a-1d43-4dd4-9fa2-2025ae91475e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:53.935068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-42780a2a-1d43-4dd4-9fa2-2025ae91475e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:53.993002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1230/2464] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:54.002542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:54.004267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:54.044877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:54.063212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbc2e4-113b-4980-b5cc-1542fbd82613 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:54.067036 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:54.067685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:54.067973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:54.068251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:54.077997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:54.079442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:54.089473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1235/2465] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:06:58.179861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee4c63-ec7b-417b-9dd0-09c25fe5afba None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:58.180302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee4c63-ec7b-417b-9dd0-09c25fe5afba None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:58.180401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee4c63-ec7b-417b-9dd0-09c25fe5afba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:58.208786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-55ee4c63-ec7b-417b-9dd0-09c25fe5afba None None] MFA Rules not processed for user `0e6fbab71db54b698572bb8a7d033063`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:06:58.210064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-55ee4c63-ec7b-417b-9dd0-09c25fe5afba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:06:58.287814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1231/2466] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:06:58.292785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:58.303766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4a97d-b7d4-46d5-a4e9-d4e01a5a3d1d service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:58.310023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:58.310795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:58.311083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:58.311359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:58.313099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:58.315585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:58.346322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1236/2467] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:06:58 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:58.350338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee4c63-ec7b-417b-9dd0-09c25fe5afba None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:58.362557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee4c63-ec7b-417b-9dd0-09c25fe5afba None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:58.447573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:06:58.475110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:58.475812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:58.476100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:58.476379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:58.477299 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:58.479078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:58.509954 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1232/2468] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:06:58 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:06:58.511590 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:58.525598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52233a4f-7a85-4226-8c76-94a7293d4c6e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:58.531369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:58.532383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:58.532649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:58.532961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:58.534027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:58.538824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:58.554965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1237/2469] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:06:58.558895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:58.574746 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68852944-8c2b-43c9-98b6-7e0bda67c3b4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:58.589737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:58.590756 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:58.591112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:58.591463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] PATH_INFO: `/v3/projects/efc03348fea54514a295e5a980912ece/users/0eefc98bb5a944e19324804fcbe53bbb/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:58.610315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc03348fea54514a295e5a980912ece, user_id=0eefc98bb5a944e19324804fcbe53bbb, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:58.613276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:58.638873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1233/2470] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:58 2022] PUT /identity/v3/projects/efc03348fea54514a295e5a980912ece/users/0eefc98bb5a944e19324804fcbe53bbb/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:06:58.642916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:58.668048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a34a4a-aa5f-43c3-a1ba-579826b501d1 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:58.740441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f998b66-0731-4566-83f1-0f7e1a366ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:58.741178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f998b66-0731-4566-83f1-0f7e1a366ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:58.741467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f998b66-0731-4566-83f1-0f7e1a366ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:58.741749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f998b66-0731-4566-83f1-0f7e1a366ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:58.742484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f998b66-0731-4566-83f1-0f7e1a366ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:58.744145 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f998b66-0731-4566-83f1-0f7e1a366ea1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:58.755617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1238/2471] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:06:58.760007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:06:58.772348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db89cb-432e-47a1-a171-5793c86c4e55 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:06:58.779029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:06:58.779981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:06:58.780268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:06:58.780547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] PATH_INFO: `/v3/projects/efc03348fea54514a295e5a980912ece/users/0eefc98bb5a944e19324804fcbe53bbb/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:06:58.783159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc03348fea54514a295e5a980912ece, user_id=0eefc98bb5a944e19324804fcbe53bbb, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:06:58.789178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:06:58.802667 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1234/2472] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:06:58 2022] PUT /identity/v3/projects/efc03348fea54514a295e5a980912ece/users/0eefc98bb5a944e19324804fcbe53bbb/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:07:03.512442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceefed7-5766-45b5-8ac2-e32ec3b70f5a None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:03.512986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceefed7-5766-45b5-8ac2-e32ec3b70f5a None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:03.513202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceefed7-5766-45b5-8ac2-e32ec3b70f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:03.542221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-2ceefed7-5766-45b5-8ac2-e32ec3b70f5a None None] MFA Rules not processed for user `0eefc98bb5a944e19324804fcbe53bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:07:03.543771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2ceefed7-5766-45b5-8ac2-e32ec3b70f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:03.615844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1239/2473] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:07:03.630088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:03.632335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:03.709815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686cba4-691a-4efd-881b-655ab86f8abc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:03.715505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0652a-6589-40c3-a6b1-2ac2c90fb927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:03.716198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0652a-6589-40c3-a6b1-2ac2c90fb927 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:03.716482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0652a-6589-40c3-a6b1-2ac2c90fb927 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:03.716760 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0652a-6589-40c3-a6b1-2ac2c90fb927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:03.772710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0652a-6589-40c3-a6b1-2ac2c90fb927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:03.774364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0652a-6589-40c3-a6b1-2ac2c90fb927 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:03.784470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1235/2474] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:04.836390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:04.836985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:04.836985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:04.854335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] MFA Rules not processed for user `0e6fbab71db54b698572bb8a7d033063`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:07:04.855541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:04.862482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1240/2475] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:07:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:07:05.125420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41895f82-ae74-4980-a7c9-72c2e04a70b5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:05.125937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41895f82-ae74-4980-a7c9-72c2e04a70b5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:05.126078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41895f82-ae74-4980-a7c9-72c2e04a70b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:05.143622 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-41895f82-ae74-4980-a7c9-72c2e04a70b5 None None] MFA Rules not processed for user `9d977f609537476daa8fe8adbab9ff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:07:05.144863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-41895f82-ae74-4980-a7c9-72c2e04a70b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:05.181574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:07:05.219665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1236/2476] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:07:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:07:09.245291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:09.246883 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:09.270678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:07:09.311624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63220e91-2435-4cf1-bbae-cfd73d689f43 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:09.319509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053726f-4efc-4bb5-838c-61122716885b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:09.320190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053726f-4efc-4bb5-838c-61122716885b service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:09.320478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053726f-4efc-4bb5-838c-61122716885b service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:09.320810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053726f-4efc-4bb5-838c-61122716885b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:09.324078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5053726f-4efc-4bb5-838c-61122716885b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:09.344749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053726f-4efc-4bb5-838c-61122716885b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:09.346280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053726f-4efc-4bb5-838c-61122716885b service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:09.358881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1241/2477] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:10.842004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41895f82-ae74-4980-a7c9-72c2e04a70b5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:10.852434 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41895f82-ae74-4980-a7c9-72c2e04a70b5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:10.859850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:10.860654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:10.862752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:10.863335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:10.865557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:10.887849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:10.892140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:10.907863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1237/2478] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:14.139285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5815-fae1-4989-805b-e6cce3207731 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:14.139846 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5815-fae1-4989-805b-e6cce3207731 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:14.139964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5815-fae1-4989-805b-e6cce3207731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:14.182267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-7d9b5815-fae1-4989-805b-e6cce3207731 None None] MFA Rules not processed for user `0b8be0ab53ae48af844bc281dee5f77c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:07:14.183559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7d9b5815-fae1-4989-805b-e6cce3207731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:14.240039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1242/2479] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:07:17.221600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:17.233476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e6aed-8aa0-40d5-a80b-210ffd27242d service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:17.239303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1438b15-2eee-4b48-b757-cb8088688143 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:17.240037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1438b15-2eee-4b48-b757-cb8088688143 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:17.240390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1438b15-2eee-4b48-b757-cb8088688143 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:17.240730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1438b15-2eee-4b48-b757-cb8088688143 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:17.242696 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f1438b15-2eee-4b48-b757-cb8088688143 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:17.332432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1438b15-2eee-4b48-b757-cb8088688143 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:17.334082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1438b15-2eee-4b48-b757-cb8088688143 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:17.352727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1238/2480] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:24.233235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=51 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:07:27.350032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=58 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:07:27.635934 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5815-fae1-4989-805b-e6cce3207731 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:27.648909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5815-fae1-4989-805b-e6cce3207731 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:27.656664 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:27.657666 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:27.657985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:27.658268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:27.660068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:27.671087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:27.672773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:27.692754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1243/2481] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:30.808380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5be8-9b09-40a3-939b-2fc26d79bd86 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:30.808927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5be8-9b09-40a3-939b-2fc26d79bd86 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:30.809026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5be8-9b09-40a3-939b-2fc26d79bd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:30.828156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-06af5be8-9b09-40a3-939b-2fc26d79bd86 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:07:30.829519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-06af5be8-9b09-40a3-939b-2fc26d79bd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:30.887683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1239/2482] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:07:30.934859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:30.936298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:31.007434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8670ff-88dc-43cc-9a9f-15b18f42e36d service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:31.012905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:31.013556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:31.013845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:31.014125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:31.022196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:31.023793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:31.032750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1244/2483] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:07:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:07:31.912085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5be8-9b09-40a3-939b-2fc26d79bd86 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:31.923014 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5be8-9b09-40a3-939b-2fc26d79bd86 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:31.926758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:31.927415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:31.927703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:31.927979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:31.929810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:32.009511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:32.012733 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:32.028568 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1240/2484] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:36.167366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:36.169572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:36.288247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959c401-4305-4805-8f8a-1b06a0678a3b service swift] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:36.294702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:36.295804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:36.296316 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:36.296757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:36.299722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:36.313358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:36.315551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:36.329521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1245/2485] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:42.020858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=55 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:07:42.307412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:42.316402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e9d8-a6bf-49ea-a00b-5149f1b56a94 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:42.320257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:42.320985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:42.321227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:42.321582 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:42.323395 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:42.333695 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:42.335248 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:42.350810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1241/2486] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:46.322623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=74 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:07:47.509703 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:47.511241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:47.572317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff200596-a9c8-480b-96b7-428d503058d5 service placement] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:47.576061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:47.576713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:47.577507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:47.577507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:47.579158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:47.593862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:47.595325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:47.606781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1246/2487] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:47.672796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:47.681388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20674be-a8d2-43d9-922a-fdec1652dd51 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:47.685031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fbb92-8af8-42bf-858a-d846862b0c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:47.685658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbb92-8af8-42bf-858a-d846862b0c07 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:47.685941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbb92-8af8-42bf-858a-d846862b0c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:47.686214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fbb92-8af8-42bf-858a-d846862b0c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:47.688116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cb8fbb92-8af8-42bf-858a-d846862b0c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:47.697850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fbb92-8af8-42bf-858a-d846862b0c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:47.699293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fbb92-8af8-42bf-858a-d846862b0c07 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:47.707057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1242/2488] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:47.779229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:47.788865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8900-e63d-47f7-9135-5884ec3131f0 service glance] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:47.792992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:47.793651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:47.793936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:47.794228 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:47.795974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:47.808877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:47.810323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:47.821566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1247/2489] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:07:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:07:49.740530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5498-27b0-428d-9605-67e8598ca059 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:49.741072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5498-27b0-428d-9605-67e8598ca059 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:49.741165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5498-27b0-428d-9605-67e8598ca059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:49.758838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0e9b5498-27b0-428d-9605-67e8598ca059 None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:07:49.759963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0e9b5498-27b0-428d-9605-67e8598ca059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:07:49.767779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1243/2490] 10.222.0.72 () {58 vars in 1009 bytes} [Thu Jun 16 15:07:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:07:49.779603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:07:49.790957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4c20b-160f-4f5b-acea-25fa1c7b76a6 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:07:49.795889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:07:49.796555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:07:49.796894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:07:49.797202 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:07:49.805186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:07:49.806771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:07:49.815716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1248/2491] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:07:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:07:59.754032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=15 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:07:59.812740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=38 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:08:18.660417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b5498-27b0-428d-9605-67e8598ca059 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:18.717260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b5498-27b0-428d-9605-67e8598ca059 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:18.724129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1828f4e-2948-4031-9586-c04006720031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:18.724791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828f4e-2948-4031-9586-c04006720031 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:18.725111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828f4e-2948-4031-9586-c04006720031 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:18.725389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828f4e-2948-4031-9586-c04006720031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:18.727706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b1828f4e-2948-4031-9586-c04006720031 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:18.761491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1828f4e-2948-4031-9586-c04006720031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:18.763214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1828f4e-2948-4031-9586-c04006720031 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:18.774227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1244/2492] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:08:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:08:19.073757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:19.096575 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e4797-9818-47da-afae-0e3926044b29 service swift] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:19.100667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:19.101464 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:19.101820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:19.102169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:19.113940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:19.115502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:19.125584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1249/2493] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:08:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:08:21.464277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f395-59c4-4049-b5d8-8680bd2d0a9c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:21.464743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f395-59c4-4049-b5d8-8680bd2d0a9c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:21.464877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f395-59c4-4049-b5d8-8680bd2d0a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:21.483799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-56e9f395-59c4-4049-b5d8-8680bd2d0a9c None None] MFA Rules not processed for user `77bd2bfa6a5343f69db3af35547a261c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:08:21.485048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-56e9f395-59c4-4049-b5d8-8680bd2d0a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:08:21.492297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1245/2494] 10.222.0.72 () {58 vars in 967 bytes} [Thu Jun 16 15:08:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:08:21.505209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:08:21.517419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7a087-7649-4a51-8224-df49686d5b6f service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:08:21.522530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2013d4199e3841109b7c4661ff45b160', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:08:21.523263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:08:21.523640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:08:21.523989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:08:21.532656 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:08:21.534307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:08:21.542839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1250/2495] 10.222.0.72 () {60 vars in 1384 bytes} [Thu Jun 16 15:08:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 15:08:31.478075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=31 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:08:31.539775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=17 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:13.365210 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9f395-59c4-4049-b5d8-8680bd2d0a9c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:13.384854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9f395-59c4-4049-b5d8-8680bd2d0a9c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:13.388390 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:13.389056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:13.389342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:13.389645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:13.391397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:13.475535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:13.477010 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:13.495798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1246/2496] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:09:16.341610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:16.343207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:16.361561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867caaf-9eef-40a6-95ce-4a4c1b2a348d service swift] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:16.366804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:16.367674 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:16.367960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:16.368235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] PATH_INFO: `/v3/users/b424792631ac40f48c3f3c1529d699b3` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:16.369488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b424792631ac40f48c3f3c1529d699b3)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:16.371432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:16.417600 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b424792631ac40f48c3f3c1529d699b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.418248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b424792631ac40f48c3f3c1529d699b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.418847 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b424792631ac40f48c3f3c1529d699b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.426173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b424792631ac40f48c3f3c1529d699b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.437667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9963938b-56e5-45b1-9631-8c332f839b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b424792631ac40f48c3f3c1529d699b3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.450269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1251/2497] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:09:16 2022] DELETE /identity/v3/users/b424792631ac40f48c3f3c1529d699b3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:16.623816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:16.634684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780be7c-54f0-4cc5-83b6-442e1b8c6060 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:16.691388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:16.692255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:16.692543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:16.692822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] PATH_INFO: `/v3/projects/3c94142f5fe543e28710c9bb2916662c` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:16.694062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c94142f5fe543e28710c9bb2916662c)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:16.695967 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:16.741551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c94142f5fe543e28710c9bb2916662c', 'request_id': 'req-e4c8cdff-c744-4305-8717-522a20a024ae'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.747229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c94142f5fe543e28710c9bb2916662c', 'request_id': 'req-e4c8cdff-c744-4305-8717-522a20a024ae'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.748773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:16.755276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c94142f5fe543e28710c9bb2916662c', 'request_id': 'req-e4c8cdff-c744-4305-8717-522a20a024ae'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.766330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] The token cache is being invalidate because project 3c94142f5fe543e28710c9bb2916662c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:09:16.766612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.767227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e4c8cdff-c744-4305-8717-522a20a024ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:16.769558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1247/2498] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:09:16 2022] DELETE /identity/v3/projects/3c94142f5fe543e28710c9bb2916662c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:16.774440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecdfc-2019-42cc-b773-f91f022291f2 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:16.774730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecdfc-2019-42cc-b773-f91f022291f2 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:16.775008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecdfc-2019-42cc-b773-f91f022291f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:16.832518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-157ecdfc-2019-42cc-b773-f91f022291f2 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:16.833749 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-157ecdfc-2019-42cc-b773-f91f022291f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:16.892700 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1252/2499] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:09:16.901518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abc684-42d6-4349-b302-e76e3df59f36 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:16.901807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abc684-42d6-4349-b302-e76e3df59f36 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:16.902086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abc684-42d6-4349-b302-e76e3df59f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:16.921285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-69abc684-42d6-4349-b302-e76e3df59f36 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:16.922526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-69abc684-42d6-4349-b302-e76e3df59f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:16.930466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1248/2500] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:16.934521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecdfc-2019-42cc-b773-f91f022291f2 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:16.944094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecdfc-2019-42cc-b773-f91f022291f2 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:16.948618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:16.949299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:16.949584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:16.949859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:16.950577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:09:16.950888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:16.952489 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:16.964351 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1253/2501] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:09:16.968417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abc684-42d6-4349-b302-e76e3df59f36 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:16.978269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abc684-42d6-4349-b302-e76e3df59f36 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:16.982895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:16.983540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:16.983823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:16.984100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:16.984786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:16.986723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.011493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1249/2502] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:09:16 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:09:17.015465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.025925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510bf6a-594f-4463-9513-476a0ceb98d3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.052355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:17.084397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.085101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.085390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.085670 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.086509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.088330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.108377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1254/2503] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:09:17 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:09:17.112465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.122659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed41c8-0021-4e73-a15b-9b4339177adb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.127183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.127862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.128149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.128428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.129194 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.130791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.141123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1250/2504] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:17.145483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.155247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de242f1b-0a55-4f75-87ba-be42d74a5f45 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.160958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878cab3-d144-4e95-b110-311c3d9ba11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.161607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878cab3-d144-4e95-b110-311c3d9ba11c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.161895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878cab3-d144-4e95-b110-311c3d9ba11c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.162171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878cab3-d144-4e95-b110-311c3d9ba11c admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.162889 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878cab3-d144-4e95-b110-311c3d9ba11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.164481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878cab3-d144-4e95-b110-311c3d9ba11c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.174323 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1255/2505] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:17.178249 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.188263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e8a5a-d664-4a64-9207-2b5ca69d7477 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.193095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.193960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.194236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.194504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] PATH_INFO: `/v3/projects/651db7af7b3046f3b27c8a8279b8c47c/users/ef1fd5e4236a4818a959ad7b0cd705f4/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.209348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=651db7af7b3046f3b27c8a8279b8c47c, user_id=ef1fd5e4236a4818a959ad7b0cd705f4, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.212113 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.226284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1251/2506] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:09:17 2022] PUT /identity/v3/projects/651db7af7b3046f3b27c8a8279b8c47c/users/ef1fd5e4236a4818a959ad7b0cd705f4/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:17.236163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf8d95-adbe-4d1f-ad99-d32a813b4a66 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.236457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf8d95-adbe-4d1f-ad99-d32a813b4a66 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.236789 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf8d95-adbe-4d1f-ad99-d32a813b4a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.262989 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5bcf8d95-adbe-4d1f-ad99-d32a813b4a66 None None] MFA Rules not processed for user `ef1fd5e4236a4818a959ad7b0cd705f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:17.264134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5bcf8d95-adbe-4d1f-ad99-d32a813b4a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:17.327768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1256/2507] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:17.333057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.398294 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485e36c-9c06-4867-9842-7a348bfc7627 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.403627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.404271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.404549 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.404818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.405534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.407430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.436913 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1252/2508] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:09:17 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:09:17.440775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf8d95-adbe-4d1f-ad99-d32a813b4a66 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.451679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf8d95-adbe-4d1f-ad99-d32a813b4a66 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.514660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.515252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.515542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.515845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.516774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.519808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.545269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1257/2509] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:09:17 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:09:17.549163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.560625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b65030-e554-4583-8b31-d88253642ac4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.567900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.569522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.570107 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.570574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.572271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.574539 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.587512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1253/2510] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:17.591814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.602361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8b55f-0984-4a76-85b2-f02ac3750c17 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.607175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.608112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.608402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.608686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] PATH_INFO: `/v3/projects/4594efcb0ec14f158a1817fe02204ad4/users/c984b41630d248458c725dbb16def4c8/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.624102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4594efcb0ec14f158a1817fe02204ad4, user_id=c984b41630d248458c725dbb16def4c8, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.627100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.641270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1258/2511] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:09:17 2022] PUT /identity/v3/projects/4594efcb0ec14f158a1817fe02204ad4/users/c984b41630d248458c725dbb16def4c8/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:17.645925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.656181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbddfa-e793-4e71-9493-8621c501a2b5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.708762 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:17.715332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70281b-786f-4db6-94c2-0aae20509f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.716152 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70281b-786f-4db6-94c2-0aae20509f36 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.716612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70281b-786f-4db6-94c2-0aae20509f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.717095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70281b-786f-4db6-94c2-0aae20509f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.718359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70281b-786f-4db6-94c2-0aae20509f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.720056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70281b-786f-4db6-94c2-0aae20509f36 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.730758 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1254/2512] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:17.735131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.741306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:17.749669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408d549-50b0-4a4c-9746-1ce042e7bd9d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.754927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.755817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.756109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.756511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] PATH_INFO: `/v3/projects/4594efcb0ec14f158a1817fe02204ad4/users/c984b41630d248458c725dbb16def4c8/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.758534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4594efcb0ec14f158a1817fe02204ad4, user_id=c984b41630d248458c725dbb16def4c8, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.761265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.772019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1259/2513] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:09:17 2022] PUT /identity/v3/projects/4594efcb0ec14f158a1817fe02204ad4/users/c984b41630d248458c725dbb16def4c8/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:09:17.781895 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cb264-c254-441c-9f25-c4e652c0b103 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.782239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cb264-c254-441c-9f25-c4e652c0b103 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.782580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cb264-c254-441c-9f25-c4e652c0b103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.812126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a26cb264-c254-441c-9f25-c4e652c0b103 None None] MFA Rules not processed for user `c984b41630d248458c725dbb16def4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:17.812126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a26cb264-c254-441c-9f25-c4e652c0b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:17.880285 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1255/2514] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:17.892811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:17.894579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:17.961564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cf80e-b7b0-40a4-b65d-8e05b4c21a68 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:17.965214 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e93d5b-7a20-47b6-bdef-cdb44e7e6daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:17.965865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e93d5b-7a20-47b6-bdef-cdb44e7e6daf service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:17.966149 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e93d5b-7a20-47b6-bdef-cdb44e7e6daf service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:17.966430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e93d5b-7a20-47b6-bdef-cdb44e7e6daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:17.975719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e93d5b-7a20-47b6-bdef-cdb44e7e6daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:17.977223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e93d5b-7a20-47b6-bdef-cdb44e7e6daf service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:17.987511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1260/2515] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:09:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:09:18.381980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cb264-c254-441c-9f25-c4e652c0b103 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:18.444468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cb264-c254-441c-9f25-c4e652c0b103 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:18.449040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:18.449920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:18.450206 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:18.450486 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] PATH_INFO: `/v3/users/ef1fd5e4236a4818a959ad7b0cd705f4` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:18.451673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1fd5e4236a4818a959ad7b0cd705f4)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:18.453656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:18.500241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1fd5e4236a4818a959ad7b0cd705f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:18.500921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1fd5e4236a4818a959ad7b0cd705f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:18.507713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1fd5e4236a4818a959ad7b0cd705f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:18.508330 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1fd5e4236a4818a959ad7b0cd705f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:18.515609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1fd5e4236a4818a959ad7b0cd705f4'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:18.528633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:18.534825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1256/2516] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:09:18 2022] DELETE /identity/v3/users/ef1fd5e4236a4818a959ad7b0cd705f4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:18.540019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:18.540559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:18.540701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:18.558772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:18.559971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:18.619049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1261/2517] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:18.628812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:18.630720 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:18.699072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2488f74-1ec9-41e0-a6a1-147c292b7de4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:18.702861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:18.703511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:18.703796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:18.704104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:18.713732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:18.715162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:18.725381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1257/2518] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:09:19.007057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:19.008502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:19.022138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc5728-e3e2-4163-96bc-baf96152fc96 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:19.026902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:19.027757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.028039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.028314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] PATH_INFO: `/v3/projects/651db7af7b3046f3b27c8a8279b8c47c` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.029526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=651db7af7b3046f3b27c8a8279b8c47c)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:19.031548 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:19.078823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651db7af7b3046f3b27c8a8279b8c47c', 'request_id': 'req-dd15aa81-ab93-4128-a1c6-175e946e9877'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.084796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651db7af7b3046f3b27c8a8279b8c47c', 'request_id': 'req-dd15aa81-ab93-4128-a1c6-175e946e9877'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.091510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651db7af7b3046f3b27c8a8279b8c47c', 'request_id': 'req-dd15aa81-ab93-4128-a1c6-175e946e9877'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.102408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] The token cache is being invalidate because project 651db7af7b3046f3b27c8a8279b8c47c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:09:19.102705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.103322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.104949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1262/2519] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:09:19 2022] DELETE /identity/v3/projects/651db7af7b3046f3b27c8a8279b8c47c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:19.108794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:19.118401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20715b73-3885-402f-a90f-85c3aa56ce26 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:19.176592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:19.177502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.177784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.178059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] PATH_INFO: `/v3/users/c984b41630d248458c725dbb16def4c8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.179233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c984b41630d248458c725dbb16def4c8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:19.181201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:19.226853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c984b41630d248458c725dbb16def4c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.227511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c984b41630d248458c725dbb16def4c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.234233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c984b41630d248458c725dbb16def4c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.234849 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c984b41630d248458c725dbb16def4c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.242576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-3fb5efc6-0f21-4e56-958e-91627db3580e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c984b41630d248458c725dbb16def4c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.260441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1258/2520] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:09:19 2022] DELETE /identity/v3/users/c984b41630d248458c725dbb16def4c8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:19.534920 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:19.536488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:19.553488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15aa81-ab93-4128-a1c6-175e946e9877 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:19.558358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:19.613693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:19.614498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.614768 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.615048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] PATH_INFO: `/v3/projects/4594efcb0ec14f158a1817fe02204ad4` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.616222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] RBAC: Authorizing `identity:delete_project(project_id=4594efcb0ec14f158a1817fe02204ad4)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:19.618253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:19.665375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4594efcb0ec14f158a1817fe02204ad4', 'request_id': 'req-ba084a3d-94d3-478f-a40a-43ae81eadede'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.671175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4594efcb0ec14f158a1817fe02204ad4', 'request_id': 'req-ba084a3d-94d3-478f-a40a-43ae81eadede'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.679137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4594efcb0ec14f158a1817fe02204ad4', 'request_id': 'req-ba084a3d-94d3-478f-a40a-43ae81eadede'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.690824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] The token cache is being invalidate because project 4594efcb0ec14f158a1817fe02204ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:09:19.691114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.691747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ba084a3d-94d3-478f-a40a-43ae81eadede admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:09:19.693335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1263/2521] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:09:19 2022] DELETE /identity/v3/projects/4594efcb0ec14f158a1817fe02204ad4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:19.698311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbae744-4d74-48e0-aa9e-b1247c745a5f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.698607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbae744-4d74-48e0-aa9e-b1247c745a5f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.698885 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbae744-4d74-48e0-aa9e-b1247c745a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.716355 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-dcbae744-4d74-48e0-aa9e-b1247c745a5f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:19.717584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dcbae744-4d74-48e0-aa9e-b1247c745a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:19.722279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:19.788303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1259/2522] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:19.795954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b09da-02b8-43a0-b92c-e721fc0598b3 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.796268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b09da-02b8-43a0-b92c-e721fc0598b3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.796547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b09da-02b8-43a0-b92c-e721fc0598b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.816189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-da1b09da-02b8-43a0-b92c-e721fc0598b3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:19.817426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-da1b09da-02b8-43a0-b92c-e721fc0598b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:19.825158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1264/2523] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:19.829427 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbae744-4d74-48e0-aa9e-b1247c745a5f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:19.839195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbae744-4d74-48e0-aa9e-b1247c745a5f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:19.843740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:19.844394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.844679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.845006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.845734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:09:19.846693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:19.848314 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:19.859858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1260/2524] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:09:19.864773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b09da-02b8-43a0-b92c-e721fc0598b3 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:19.873948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b09da-02b8-43a0-b92c-e721fc0598b3 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:19.878660 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:19.879343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.879628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.879906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.880594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:19.882543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:19.909983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1265/2525] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:09:19 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:09:19.913937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:19.925246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68dca08-af3d-4b74-8097-b97ed71aeeeb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:19.985479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:19.986148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:19.986430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:19.986734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:19.987431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:19.989374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.011911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1261/2526] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:09:19 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:09:20.013231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.024767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44039951-43b5-4682-888b-8cce0407665f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.029461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.030139 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.030410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.030686 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.031409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.033460 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.045515 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1266/2527] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:20.049923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.061455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbfe0b-cab1-4bae-8ea4-2f9ad91fe656 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.068604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e185b84-50d1-4ec7-aa5f-840d06d8c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.069304 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185b84-50d1-4ec7-aa5f-840d06d8c512 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.069787 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185b84-50d1-4ec7-aa5f-840d06d8c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.070032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e185b84-50d1-4ec7-aa5f-840d06d8c512 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.070863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e185b84-50d1-4ec7-aa5f-840d06d8c512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.072510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e185b84-50d1-4ec7-aa5f-840d06d8c512 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.085590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1262/2528] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:20.090158 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.101564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a761e0-22ba-437d-b4be-ec446d2b2b6d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.106704 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.107587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.107878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.108157 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] PATH_INFO: `/v3/projects/283cf963eae0439fb70286067d3807a5/users/c61dd17faf82456eae11676ca7baa198/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.124330 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=283cf963eae0439fb70286067d3807a5, user_id=c61dd17faf82456eae11676ca7baa198, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.127138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.148909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1267/2529] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:09:20 2022] PUT /identity/v3/projects/283cf963eae0439fb70286067d3807a5/users/c61dd17faf82456eae11676ca7baa198/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:20.158283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c116c9-08c9-48b3-85db-9b0235189db5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.158617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c116c9-08c9-48b3-85db-9b0235189db5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.159459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c116c9-08c9-48b3-85db-9b0235189db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.188599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-66c116c9-08c9-48b3-85db-9b0235189db5 None None] MFA Rules not processed for user `c61dd17faf82456eae11676ca7baa198`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:20.189935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-66c116c9-08c9-48b3-85db-9b0235189db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:20.258896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1263/2530] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:20.263514 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.297826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:20.327570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927b23e-9dd7-4d45-8c1e-3e2267b76272 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.333212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.333882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.334171 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.334454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.335159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.337122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.361225 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1268/2531] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:09:20 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:09:20.365225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c116c9-08c9-48b3-85db-9b0235189db5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.371681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:20.376076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c116c9-08c9-48b3-85db-9b0235189db5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.433128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.433792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.434071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.434373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.435050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.436981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.456627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1264/2532] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:09:20 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:09:20.460626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.470935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8c4cf-bb72-45c0-99a3-6d4d240c2464 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.475595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.476251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.476540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.476822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.477610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.479210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.489524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1269/2533] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:20.493517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.503653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27baba-4af6-4193-ac5e-affc1e1e081d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.508143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.509172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.509453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.509751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] PATH_INFO: `/v3/projects/f4e2c96cb16c454187048bfe22a191d5/users/8f4d3e59f9104ec49564e00a74e636c8/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.524566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e2c96cb16c454187048bfe22a191d5, user_id=8f4d3e59f9104ec49564e00a74e636c8, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.527532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.541491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1265/2534] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:09:20 2022] PUT /identity/v3/projects/f4e2c96cb16c454187048bfe22a191d5/users/8f4d3e59f9104ec49564e00a74e636c8/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:20.545946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.555987 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3e49a-ee81-46e1-a86b-9729745dee4d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.611493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edf6220-cca1-47f9-bd35-ef37d83ec6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.612188 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6220-cca1-47f9-bd35-ef37d83ec6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.612482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6220-cca1-47f9-bd35-ef37d83ec6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.612756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edf6220-cca1-47f9-bd35-ef37d83ec6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.613502 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf6220-cca1-47f9-bd35-ef37d83ec6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.615106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edf6220-cca1-47f9-bd35-ef37d83ec6e7 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.624604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1270/2535] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:09:20.629358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.639337 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f807fe2-062d-4a3f-85a8-9e20a367dbe7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.644637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.646960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.647254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.647532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] PATH_INFO: `/v3/projects/f4e2c96cb16c454187048bfe22a191d5/users/8f4d3e59f9104ec49564e00a74e636c8/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.649525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e2c96cb16c454187048bfe22a191d5, user_id=8f4d3e59f9104ec49564e00a74e636c8, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.652139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.662382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1266/2536] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:09:20 2022] PUT /identity/v3/projects/f4e2c96cb16c454187048bfe22a191d5/users/8f4d3e59f9104ec49564e00a74e636c8/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:09:20.673230 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.673527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.673809 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.700665 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] MFA Rules not processed for user `8f4d3e59f9104ec49564e00a74e636c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:09:20.701902 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:20.768925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1271/2537] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:09:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:09:20.780798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:20.782543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:20.852965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a10669-f697-4a71-b95b-4848769529ff admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:20.856609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:20.857283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:20.857572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:20.857847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:20.913779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:20.915291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:20.919850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:20.926074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1267/2538] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:09:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:09:29.636032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:29.637456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:29.649793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a46864-0e2b-4608-bc7d-4c1bf402ee73 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:29.653354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:29.653992 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:29.654278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:29.654561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:29.663752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:29.665220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:29.679532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1272/2539] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:09:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:09:39.670424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:44.722188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:44.723962 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:44.853654 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f3ac-eff1-43bc-8b20-895f3de4e4fe service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:44.860348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:44.861229 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:44.861532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:44.861811 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:44.864081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:44.878225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:44.880579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:44.896444 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1268/2540] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:09:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:09:54.890120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=46 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:09:59.306308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:09:59.308400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:59.380025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f164400-9c60-44b6-8f99-f7fc1c1d3c49 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:09:59.384289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22daea7-c020-4aff-aeea-5be6ef40b72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:09:59.385056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22daea7-c020-4aff-aeea-5be6ef40b72b service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:09:59.385397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22daea7-c020-4aff-aeea-5be6ef40b72b service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:09:59.386181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22daea7-c020-4aff-aeea-5be6ef40b72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:09:59.387974 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b22daea7-c020-4aff-aeea-5be6ef40b72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:09:59.478758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22daea7-c020-4aff-aeea-5be6ef40b72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:09:59.480237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22daea7-c020-4aff-aeea-5be6ef40b72b service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:09:59.491287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1273/2541] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:02.911634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:02.927244 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9f32f-4247-4950-a148-1094c801fc21 service nova] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:02.939200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:02.940094 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:02.940603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:02.940740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] PATH_INFO: `/v3/users/c61dd17faf82456eae11676ca7baa198` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:02.941960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61dd17faf82456eae11676ca7baa198)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:02.944497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:03.005637 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61dd17faf82456eae11676ca7baa198'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.006409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61dd17faf82456eae11676ca7baa198'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.017536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61dd17faf82456eae11676ca7baa198'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.018503 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61dd17faf82456eae11676ca7baa198'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.035070 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61dd17faf82456eae11676ca7baa198'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.063987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1269/2542] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:02 2022] DELETE /identity/v3/users/c61dd17faf82456eae11676ca7baa198 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:03.068705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:03.069589 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:03.069877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:03.090434 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:03.091810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:03.173576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1274/2543] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:03.184366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:03.186136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:03.257404 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e795b86-55ff-4bb0-9a2d-bc4fdea374dc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:03.261087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:03.262138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:03.262138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:03.262353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:03.273096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:03.274961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:03.296386 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1270/2544] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:03.396800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:03.398449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:03.429601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c17233-ce1f-4c9f-b4db-f14ff09957f6 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:03.438462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:03.439519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:03.439814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:03.440095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] PATH_INFO: `/v3/users/0b8be0ab53ae48af844bc281dee5f77c` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:03.441368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8be0ab53ae48af844bc281dee5f77c)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:03.444132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:03.534325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:03.536195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8be0ab53ae48af844bc281dee5f77c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.538009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8be0ab53ae48af844bc281dee5f77c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.539105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8be0ab53ae48af844bc281dee5f77c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.549348 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8be0ab53ae48af844bc281dee5f77c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.566890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8be0ab53ae48af844bc281dee5f77c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.587388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1275/2545] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:03 2022] DELETE /identity/v3/users/0b8be0ab53ae48af844bc281dee5f77c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:03.702550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:03.704093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:03.718606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218a6cd-7c09-49e0-b2a8-ffe9ce8c2e5b service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:03.791608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:03.792487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:03.792773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:03.793082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] PATH_INFO: `/v3/projects/283cf963eae0439fb70286067d3807a5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:03.794471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=283cf963eae0439fb70286067d3807a5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:03.796429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:03.806527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:03.807948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:03.820533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:03.828619 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115f928-74ed-46a9-bc64-d1912fa1e365 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:03.836212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:03.837859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:03.838269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:03.838773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] PATH_INFO: `/v3/projects/2940b81a86c2401db1aa432d9a0735eb` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:03.840687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2940b81a86c2401db1aa432d9a0735eb)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:03.847319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:03.874221 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '283cf963eae0439fb70286067d3807a5', 'request_id': 'req-efa8dd3c-c277-4674-b61e-04dc4d21a793'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.883816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '283cf963eae0439fb70286067d3807a5', 'request_id': 'req-efa8dd3c-c277-4674-b61e-04dc4d21a793'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.891785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '283cf963eae0439fb70286067d3807a5', 'request_id': 'req-efa8dd3c-c277-4674-b61e-04dc4d21a793'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.903207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2940b81a86c2401db1aa432d9a0735eb', 'request_id': 'req-8a97b32f-a9a7-476d-b321-f31110236119'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.904143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] The token cache is being invalidate because project 283cf963eae0439fb70286067d3807a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:03.904438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.905289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.906754 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1271/2546] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:03 2022] DELETE /identity/v3/projects/283cf963eae0439fb70286067d3807a5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:03.909567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2940b81a86c2401db1aa432d9a0735eb', 'request_id': 'req-8a97b32f-a9a7-476d-b321-f31110236119'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.910779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:03.912128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:03.917912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2940b81a86c2401db1aa432d9a0735eb', 'request_id': 'req-8a97b32f-a9a7-476d-b321-f31110236119'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.930422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] The token cache is being invalidate because project 2940b81a86c2401db1aa432d9a0735eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:03.930899 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.931716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-8a97b32f-a9a7-476d-b321-f31110236119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:03.933328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1276/2547] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:03 2022] DELETE /identity/v3/projects/2940b81a86c2401db1aa432d9a0735eb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:03.937942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277d015-fbbf-45bc-9ec5-1d378cf3cd52 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:03.938342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277d015-fbbf-45bc-9ec5-1d378cf3cd52 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:03.938688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277d015-fbbf-45bc-9ec5-1d378cf3cd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:03.957568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-f277d015-fbbf-45bc-9ec5-1d378cf3cd52 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:03.959364 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f277d015-fbbf-45bc-9ec5-1d378cf3cd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:03.979462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8dd3c-c277-4674-b61e-04dc4d21a793 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.028076 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1277/2548] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:04.035823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079766cc-5313-429f-82ae-cf1b464cc84e None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.036295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079766cc-5313-429f-82ae-cf1b464cc84e None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.036436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079766cc-5313-429f-82ae-cf1b464cc84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.046263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.047149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.047419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.047688 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] PATH_INFO: `/v3/users/8f4d3e59f9104ec49564e00a74e636c8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.049013 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4d3e59f9104ec49564e00a74e636c8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.051065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.055694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-079766cc-5313-429f-82ae-cf1b464cc84e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:04.056932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-079766cc-5313-429f-82ae-cf1b464cc84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.065540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1278/2549] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:04.070518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079766cc-5313-429f-82ae-cf1b464cc84e None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.079988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079766cc-5313-429f-82ae-cf1b464cc84e None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.085606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.086268 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.086550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.086831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.087553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:04.087865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.089500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.099371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4d3e59f9104ec49564e00a74e636c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.100059 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4d3e59f9104ec49564e00a74e636c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.106062 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1279/2550] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:10:04.106214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:04.107083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.108032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4d3e59f9104ec49564e00a74e636c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.108819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4d3e59f9104ec49564e00a74e636c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.109017 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.117507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4d3e59f9104ec49564e00a74e636c8'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.122524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c32cdc-99e8-4bdf-b0ae-fc469319cf51 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.127006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.127654 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.127954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.128213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.128993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.131458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.136209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1272/2551] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:03 2022] DELETE /identity/v3/users/8f4d3e59f9104ec49564e00a74e636c8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:04.160919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1280/2552] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:04.162602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.232740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5d25a-869d-433b-8447-ff04b362ad51 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.292598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.293325 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.293609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.293883 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.294586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.296517 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.321417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1273/2553] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:04.325799 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.334340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:04.338528 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5834f3-b066-480c-b6b7-ebc728c94071 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.345570 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.346432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.347061 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.347569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.348897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.351246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.362299 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1281/2554] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:04.367046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.378445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c40648-bf56-473d-a450-34823982a167 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.392521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.393697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.394067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.394411 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.395461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.397405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.409208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1274/2555] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:04.413701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.424411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0d25e-a3bf-48eb-9c7e-76097573ef34 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.429863 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956bce0-67d3-46aa-b15d-858107b11f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.431422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bce0-67d3-46aa-b15d-858107b11f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.431754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bce0-67d3-46aa-b15d-858107b11f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.432037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bce0-67d3-46aa-b15d-858107b11f93 admin admin] PATH_INFO: `/v3/projects/806353d0bb7d4b5eb5d1939c58d9ab5c/users/6b2b8e2cea7247c28500df9e9141b2d7/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.442364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.444819 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.455120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956bce0-67d3-46aa-b15d-858107b11f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=806353d0bb7d4b5eb5d1939c58d9ab5c, user_id=6b2b8e2cea7247c28500df9e9141b2d7, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.458051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bad39-6a67-4498-b60f-ec030b4bf5b8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.458447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956bce0-67d3-46aa-b15d-858107b11f93 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.463459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.464575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.464963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.465331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] PATH_INFO: `/v3/projects/f4e2c96cb16c454187048bfe22a191d5` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.467202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e2c96cb16c454187048bfe22a191d5)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.469076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.476175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1282/2556] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:04 2022] PUT /identity/v3/projects/806353d0bb7d4b5eb5d1939c58d9ab5c/users/6b2b8e2cea7247c28500df9e9141b2d7/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:04.487675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.488580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.488969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.521168 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e2c96cb16c454187048bfe22a191d5', 'request_id': 'req-c7a99e17-8dbe-4961-a642-c749373e055f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.524058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] MFA Rules not processed for user `6b2b8e2cea7247c28500df9e9141b2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:04.524477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:04.525334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.530748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e2c96cb16c454187048bfe22a191d5', 'request_id': 'req-c7a99e17-8dbe-4961-a642-c749373e055f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.539348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e2c96cb16c454187048bfe22a191d5', 'request_id': 'req-c7a99e17-8dbe-4961-a642-c749373e055f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.552556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] The token cache is being invalidate because project f4e2c96cb16c454187048bfe22a191d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:04.552963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.553624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c7a99e17-8dbe-4961-a642-c749373e055f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:04.555179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1275/2557] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:04 2022] DELETE /identity/v3/projects/f4e2c96cb16c454187048bfe22a191d5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:04.559748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167df7-98c5-4235-9478-381cd206dddf None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.560092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167df7-98c5-4235-9478-381cd206dddf None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.560351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167df7-98c5-4235-9478-381cd206dddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.579499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-50167df7-98c5-4235-9478-381cd206dddf None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:04.580711 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-50167df7-98c5-4235-9478-381cd206dddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.613009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1283/2558] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:04.615371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.616785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.685149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1276/2559] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:04.692083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.693677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.694224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.724043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24df317-0608-4653-b308-05a73b29d7d4 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.730276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.730939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.731223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.731500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.731779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:04.733120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.740629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.742371 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.742935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1277/2560] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:04.743541 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.744800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.752349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1284/2561] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:04.753822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.764892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9126f6-9ba3-4079-8ab6-2aaaceed0629 service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.776250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.777067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.777350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.778097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.778419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:04.778652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.780244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.795962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1285/2562] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:04.800557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.818956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074f7b4-7247-443f-8e14-25b343f4190b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.820130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964fe61-52dc-4e9a-9f0a-9fceb26673cb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.824012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.824665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.824980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.825258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.827076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:04.834433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.838662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.840045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.840176 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.841380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.841624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.842313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.845029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:04.853231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1278/2563] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:04.889517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1286/2564] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:04.889737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:04.901171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678bc21-4353-4fe3-9d34-21d11a5fc2c9 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:04.972020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:04.973712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:04.977278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:04.977766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:04.979095 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:04.981896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.010389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1279/2565] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:04 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:05.014040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.034092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f5156-766d-42fa-ac5d-f6b1db9ecb3f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.044554 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.045678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.045971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.046255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.047220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.048950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.052360 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:05.069513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1287/2566] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:05.070507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.089365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d63134-4df1-4d8c-a970-248c2beab638 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.098063 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4da727-d76f-4ba5-b6d9-f9079f4a15ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.098855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4da727-d76f-4ba5-b6d9-f9079f4a15ee admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.100359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4da727-d76f-4ba5-b6d9-f9079f4a15ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.101051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4da727-d76f-4ba5-b6d9-f9079f4a15ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.101896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4da727-d76f-4ba5-b6d9-f9079f4a15ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.104315 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4da727-d76f-4ba5-b6d9-f9079f4a15ee admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.107713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:05.133676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.144497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1280/2567] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:05.153222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cfae9-cac6-41d6-9bf5-78f6cf8a94a8 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.159792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.162742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.163367 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.163979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] PATH_INFO: `/v3/projects/6e8557bf30564fe09f4ed7ff8efd241f/users/deda392de1dc48abb29fab0e06309313/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.185802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8557bf30564fe09f4ed7ff8efd241f, user_id=deda392de1dc48abb29fab0e06309313, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.190404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.238441 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1288/2568] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:05 2022] PUT /identity/v3/projects/6e8557bf30564fe09f4ed7ff8efd241f/users/deda392de1dc48abb29fab0e06309313/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:05.240632 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531adbae-53bd-450a-aa1f-dd43f2667118 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.241142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531adbae-53bd-450a-aa1f-dd43f2667118 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.241426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531adbae-53bd-450a-aa1f-dd43f2667118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.306551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-531adbae-53bd-450a-aa1f-dd43f2667118 None None] MFA Rules not processed for user `deda392de1dc48abb29fab0e06309313`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:05.308414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-531adbae-53bd-450a-aa1f-dd43f2667118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:05.414559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1281/2569] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:05.418885 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.490616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cf2b4-4ac0-4b72-a58c-99c86ba1301d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.496762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.498164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.498508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.498796 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.499505 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.501926 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.536773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1289/2570] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:05 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:05.540879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531adbae-53bd-450a-aa1f-dd43f2667118 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.553270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531adbae-53bd-450a-aa1f-dd43f2667118 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.618181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.618966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.619205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.619483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.620187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.622188 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.643205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1282/2571] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:05 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:05.646942 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.658060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40cdcf-9b51-49f6-82e7-29bcc70db901 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.664324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.665321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.665609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.665891 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.666618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.668223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.681539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1290/2572] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:05.685245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.698391 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b341e-bf6a-484e-afbd-0713d25fa9d7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.703149 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516f1ca-6218-43b4-b0ff-791ee25f1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.703818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f1ca-6218-43b4-b0ff-791ee25f1fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.704101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f1ca-6218-43b4-b0ff-791ee25f1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.704376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f1ca-6218-43b4-b0ff-791ee25f1fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.705137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516f1ca-6218-43b4-b0ff-791ee25f1fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.706732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516f1ca-6218-43b4-b0ff-791ee25f1fd2 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.717957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1283/2573] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:05.722151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.732694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91891910-1c1e-4df3-a074-c255d76c21f0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.737359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.738244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.738798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.738798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] PATH_INFO: `/v3/projects/2580748d5071406098e2f1710e4eab93/users/ded5e005037f400aa9c29eaefdd94769/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.754923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2580748d5071406098e2f1710e4eab93, user_id=ded5e005037f400aa9c29eaefdd94769, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.758170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:05.782104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1291/2574] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:05 2022] PUT /identity/v3/projects/2580748d5071406098e2f1710e4eab93/users/ded5e005037f400aa9c29eaefdd94769/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:05.783413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a18a3-f100-4c61-9642-0cc3347b7069 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.783715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a18a3-f100-4c61-9642-0cc3347b7069 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.783993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a18a3-f100-4c61-9642-0cc3347b7069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.819204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a31a18a3-f100-4c61-9642-0cc3347b7069 None None] MFA Rules not processed for user `ded5e005037f400aa9c29eaefdd94769`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:05.820467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a31a18a3-f100-4c61-9642-0cc3347b7069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:05.885824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:05.898526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1284/2575] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:05.902975 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:05.980107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275945c-1bd5-4f62-8a7e-da517a1e4782 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:05.988278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:05.990731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:05.990731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:05.990731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:05.991385 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:05.993583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.027056 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1292/2576] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:05 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:06.031566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a18a3-f100-4c61-9642-0cc3347b7069 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:06.047601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:06.049879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:06.055281 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a18a3-f100-4c61-9642-0cc3347b7069 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:06.068343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:06.125940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:06.126785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.127545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.127933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.128590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:06.130717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.159922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1285/2577] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:06 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:06.160869 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:06.186066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def3211-0381-4e69-b6bc-3ed06a1a4e99 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:06.199357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:06.200328 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.200625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.200934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.201727 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:06.203775 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.217018 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11732c0-135c-4674-898d-37f81265c308 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:06.223253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b8661ae3402445c9c027e739983f26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:06.224746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.225281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.225626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.228553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:06.228722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1286/2578] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:06.233214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:06.249184 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eefcfe-82df-4721-9482-5e04ec183828 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:06.255853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:06.256366 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:06.257502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.258009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.258499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.258737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] PATH_INFO: `/v3/projects/72919baf4182444996dbf3920b53479d/users/e839fc95db0e4bc29fa3eeeb3c013a0a/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.287471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72919baf4182444996dbf3920b53479d, user_id=e839fc95db0e4bc29fa3eeeb3c013a0a, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:06.294576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1293/2579] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:06.296786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.327835 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1287/2580] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:06 2022] PUT /identity/v3/projects/72919baf4182444996dbf3920b53479d/users/e839fc95db0e4bc29fa3eeeb3c013a0a/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:06.331595 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:06.355421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bb34f-4795-4470-9171-8e440f3c27b4 service nova] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:06.485854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a40df-9d47-4100-90de-acf0294927b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:06.486782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a40df-9d47-4100-90de-acf0294927b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.487306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a40df-9d47-4100-90de-acf0294927b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.487678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a40df-9d47-4100-90de-acf0294927b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.488525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a40df-9d47-4100-90de-acf0294927b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:06.491414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a40df-9d47-4100-90de-acf0294927b5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.518952 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1294/2581] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:06.525428 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:06.554686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9c7b5-db98-496c-9016-2ac4d80e1ae7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:06.560129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:06.560948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.561241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.561516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] PATH_INFO: `/v3/projects/72919baf4182444996dbf3920b53479d/users/e839fc95db0e4bc29fa3eeeb3c013a0a/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.564538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=72919baf4182444996dbf3920b53479d, user_id=e839fc95db0e4bc29fa3eeeb3c013a0a, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:06.567979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.582995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1288/2582] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:06 2022] PUT /identity/v3/projects/72919baf4182444996dbf3920b53479d/users/e839fc95db0e4bc29fa3eeeb3c013a0a/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:06.594208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.594615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.594966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.641324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] MFA Rules not processed for user `e839fc95db0e4bc29fa3eeeb3c013a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:06.644098 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:06.751705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1295/2583] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:06.763262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:06.765042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:06.848125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e5258-42c9-4668-a168-b891a245f1bd admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:06.851896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:06.852547 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:06.852832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:06.853372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:06.863833 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:06.865019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:06.876511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1289/2584] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:07.534747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:07.536301 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:07.556988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40aa5de-5ee8-474d-ba84-7c3085b83707 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:07.569284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee499a88-a590-4802-8fe6-ee39278a0983 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:07.569831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee499a88-a590-4802-8fe6-ee39278a0983 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:07.570542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee499a88-a590-4802-8fe6-ee39278a0983 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:07.570542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee499a88-a590-4802-8fe6-ee39278a0983 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:07.616689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:07.753350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee499a88-a590-4802-8fe6-ee39278a0983 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:07.759865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee499a88-a590-4802-8fe6-ee39278a0983 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:07.789852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1296/2585] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:07.875118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:07.885776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:07.952732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:08.009964 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bf102-1356-4203-b602-89584c5a66f1 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:08.021624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:08.022767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:08.023047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:08.023320 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] PATH_INFO: `/v3/users/6b2b8e2cea7247c28500df9e9141b2d7` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:08.025915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b2b8e2cea7247c28500df9e9141b2d7)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:08.027903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:08.096454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b2b8e2cea7247c28500df9e9141b2d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.097193 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b2b8e2cea7247c28500df9e9141b2d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.105959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b2b8e2cea7247c28500df9e9141b2d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.106809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b2b8e2cea7247c28500df9e9141b2d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.117224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b2b8e2cea7247c28500df9e9141b2d7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.143730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1290/2586] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:07 2022] DELETE /identity/v3/users/6b2b8e2cea7247c28500df9e9141b2d7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:08.145556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:08.146047 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:08.146406 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:08.176705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:08.178525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:08.305884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1297/2587] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:08.316050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:08.317488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:08.446203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d569e5-0186-4cde-b256-8afda7bf50a6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:08.450111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3b6d2-93b1-4de8-9c8f-b6d36ad6d066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:08.450769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3b6d2-93b1-4de8-9c8f-b6d36ad6d066 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:08.451067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3b6d2-93b1-4de8-9c8f-b6d36ad6d066 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:08.451397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3b6d2-93b1-4de8-9c8f-b6d36ad6d066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:08.466069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3b6d2-93b1-4de8-9c8f-b6d36ad6d066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:08.468183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3b6d2-93b1-4de8-9c8f-b6d36ad6d066 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:08.489624 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1291/2588] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:08.813467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:08.815477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:08.827878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14601cee-a6ee-4c90-9a5c-fdb96880f78b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:08.833931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:08.835109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:08.835457 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:08.835738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] PATH_INFO: `/v3/projects/806353d0bb7d4b5eb5d1939c58d9ab5c` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:08.837019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=806353d0bb7d4b5eb5d1939c58d9ab5c)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:08.839064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:08.904100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806353d0bb7d4b5eb5d1939c58d9ab5c', 'request_id': 'req-76031b3d-dd6a-49f1-a326-aa02b8c4c677'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.910616 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806353d0bb7d4b5eb5d1939c58d9ab5c', 'request_id': 'req-76031b3d-dd6a-49f1-a326-aa02b8c4c677'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.921358 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806353d0bb7d4b5eb5d1939c58d9ab5c', 'request_id': 'req-76031b3d-dd6a-49f1-a326-aa02b8c4c677'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.937269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] The token cache is being invalidate because project 806353d0bb7d4b5eb5d1939c58d9ab5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:08.938049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.939133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-76031b3d-dd6a-49f1-a326-aa02b8c4c677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:08.941922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1298/2589] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:08 2022] DELETE /identity/v3/projects/806353d0bb7d4b5eb5d1939c58d9ab5c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:08.946732 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f590db-a3f2-4197-925f-3fea50319bc5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:08.947257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f590db-a3f2-4197-925f-3fea50319bc5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:08.947376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f590db-a3f2-4197-925f-3fea50319bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:08.983500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-18f590db-a3f2-4197-925f-3fea50319bc5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:08.989752 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-18f590db-a3f2-4197-925f-3fea50319bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:09.074769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:09.093080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1292/2590] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:09.101109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc12f-515d-435c-8baa-928b7d41462b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.101399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc12f-515d-435c-8baa-928b7d41462b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.101685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fc12f-515d-435c-8baa-928b7d41462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.120472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-868fc12f-515d-435c-8baa-928b7d41462b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:09.121793 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-868fc12f-515d-435c-8baa-928b7d41462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:09.130906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1299/2591] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:09.135170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f590db-a3f2-4197-925f-3fea50319bc5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.145402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f590db-a3f2-4197-925f-3fea50319bc5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.150817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.151558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.151848 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.152140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.152921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:09.153205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.154821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.166792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1293/2592] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:09.171052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fc12f-515d-435c-8baa-928b7d41462b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.181798 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fc12f-515d-435c-8baa-928b7d41462b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.186982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.187650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.187935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.188212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.188959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.190924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.221105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1300/2593] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:09 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:09.222519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.232366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657eb96-210b-4499-a591-314bd724411c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.293580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.294344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.294697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.295046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.295803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.297831 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.337683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1294/2594] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:09 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:09.341876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.357382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ff476-636a-4782-8d62-9026600ba17a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.374797 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.376081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.376527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.377052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.378134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.380213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.384650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:09.423596 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.430782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1301/2595] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:09.438039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f3ead-111c-437f-b427-97659ff21063 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.451124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb19529-7023-4f5a-b9a9-bd5671601f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.451124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19529-7023-4f5a-b9a9-bd5671601f22 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.451124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19529-7023-4f5a-b9a9-bd5671601f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.451124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb19529-7023-4f5a-b9a9-bd5671601f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.451543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb19529-7023-4f5a-b9a9-bd5671601f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.451543 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb19529-7023-4f5a-b9a9-bd5671601f22 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.462497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1295/2596] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:09.466563 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.478450 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d30157-92d0-43ec-9601-c024538569f3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.485597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.487110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.487580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.487955 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] PATH_INFO: `/v3/projects/c86fdfb15b0c4925847fea258a5cc532/users/b6248d87f0504e8f9fe5f1269ac1cef3/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.520191 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86fdfb15b0c4925847fea258a5cc532, user_id=b6248d87f0504e8f9fe5f1269ac1cef3, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.523538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.541750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1302/2597] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:09 2022] PUT /identity/v3/projects/c86fdfb15b0c4925847fea258a5cc532/users/b6248d87f0504e8f9fe5f1269ac1cef3/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:09.547288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.548276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.548651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.573680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:09.574929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:09.650243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1296/2598] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:09.650803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.650803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:09.657465 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.659087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:09.673235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d4de8-b6d3-4b67-909e-def7bc760531 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.678814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.679755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.680131 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.680412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] PATH_INFO: `/v3/users/ba40bcf173404e6c8b17d311b65bf780` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.682243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba40bcf173404e6c8b17d311b65bf780)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.684490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.735214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb89ed7-33e8-451a-ace2-9b01ad009125 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:09.736681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba40bcf173404e6c8b17d311b65bf780'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:09.737368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba40bcf173404e6c8b17d311b65bf780'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:09.738128 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba40bcf173404e6c8b17d311b65bf780'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:09.739311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:09.740769 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:09.740924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:09.741162 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:09.743018 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:09.746182 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba40bcf173404e6c8b17d311b65bf780'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:09.755954 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:09.757599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:09.759454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f944c06c-3832-4cda-8c02-30d367913955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba40bcf173404e6c8b17d311b65bf780'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:09.767704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1297/2599] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:09.776006 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1303/2600] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:09 2022] DELETE /identity/v3/users/ba40bcf173404e6c8b17d311b65bf780 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:09.972731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:09.974378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:09.989884 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5793813-eb1c-47f7-ae11-9fc697022a68 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.000823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:10.073731 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.074724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.075074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.075741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] PATH_INFO: `/v3/projects/2a4c67a42f8a405d84d4697a45e962ff` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.076795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4c67a42f8a405d84d4697a45e962ff)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.078979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.154409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4c67a42f8a405d84d4697a45e962ff', 'request_id': 'req-39c4fda5-bfc4-448c-8252-288b374fa466'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:10.165781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4c67a42f8a405d84d4697a45e962ff', 'request_id': 'req-39c4fda5-bfc4-448c-8252-288b374fa466'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:10.177878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4c67a42f8a405d84d4697a45e962ff', 'request_id': 'req-39c4fda5-bfc4-448c-8252-288b374fa466'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:10.195490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] The token cache is being invalidate because project 2a4c67a42f8a405d84d4697a45e962ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:10.196075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:10.197134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-39c4fda5-bfc4-448c-8252-288b374fa466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:10.199797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1298/2601] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:09 2022] DELETE /identity/v3/projects/2a4c67a42f8a405d84d4697a45e962ff => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:10.204894 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998714fe-6da5-4fff-8352-a123a680900b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.205427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998714fe-6da5-4fff-8352-a123a680900b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.205427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998714fe-6da5-4fff-8352-a123a680900b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.225521 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-998714fe-6da5-4fff-8352-a123a680900b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:10.226746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-998714fe-6da5-4fff-8352-a123a680900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:10.294212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1304/2602] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:10.304138 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05236fc-2b40-4610-afb3-46185e9ff367 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.304905 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05236fc-2b40-4610-afb3-46185e9ff367 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.305830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05236fc-2b40-4610-afb3-46185e9ff367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.329567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b05236fc-2b40-4610-afb3-46185e9ff367 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:10.330792 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b05236fc-2b40-4610-afb3-46185e9ff367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:10.344126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1299/2603] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:10.345537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998714fe-6da5-4fff-8352-a123a680900b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.357983 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998714fe-6da5-4fff-8352-a123a680900b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.363683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.364468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.365436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.365820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.366667 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:10.367079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.368876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.384204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1305/2604] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:10.388464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05236fc-2b40-4610-afb3-46185e9ff367 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.401791 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05236fc-2b40-4610-afb3-46185e9ff367 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.407160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0d808-38d0-43ac-b74b-c16c768e8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.407925 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0d808-38d0-43ac-b74b-c16c768e8a52 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.408532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0d808-38d0-43ac-b74b-c16c768e8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.408790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0d808-38d0-43ac-b74b-c16c768e8a52 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.409724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0d808-38d0-43ac-b74b-c16c768e8a52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.412028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0d808-38d0-43ac-b74b-c16c768e8a52 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.447972 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1300/2605] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:10.452275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.468969 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08184d1e-b2c1-4bb6-b307-476f70d8da51 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.471485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.471822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.472124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.503777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] MFA Rules not processed for user `b6248d87f0504e8f9fe5f1269ac1cef3`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:10.505456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:10.531577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:10.583531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.584057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.584442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.584786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.585808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.588122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.595490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:10.602680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1301/2606] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:10.607400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.610032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:10.619267 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1306/2607] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:10.623871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.641862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa7088-ff60-4cef-a0b9-c7b7f7da8d12 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.646605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22ce55-c853-4f03-8cab-f5c69ee64051 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.654788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.655916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.656592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.657286 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.657522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.657802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.658777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.660735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.662257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.662602 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.663448 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.665120 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.679325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1307/2608] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:10.684542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.696057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8dcdd-1a0b-4ae9-8fa3-39dbe2b2cbf7 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.702957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.704250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.704543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.704884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.705227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1302/2609] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:10.705794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.707414 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.709084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.728091 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1308/2610] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:10.732407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.746491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a9ea5-9ad1-48d2-b656-671df11bc825 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.801500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde2bab-1148-4d3e-8bc8-2b139497ddb9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.814208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.814921 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.815226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.815497 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.816283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.818392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.820079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711dccb-579a-43af-978e-a74f6aac7df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.821099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711dccb-579a-43af-978e-a74f6aac7df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.821314 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711dccb-579a-43af-978e-a74f6aac7df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.821568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711dccb-579a-43af-978e-a74f6aac7df0 admin admin] PATH_INFO: `/v3/projects/9bb8362c5ac74c51b00fed8b524af56f/users/b5fec9a761cf4e18972eca72eff64152/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.843770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1303/2611] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/users => generated 309 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:10.844225 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711dccb-579a-43af-978e-a74f6aac7df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb8362c5ac74c51b00fed8b524af56f, user_id=b5fec9a761cf4e18972eca72eff64152, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.847740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:10.848829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711dccb-579a-43af-978e-a74f6aac7df0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.859723 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a93d8-fdb5-4a66-8e0d-9fede4f4335e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:10.873132 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1309/2612] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:10 2022] PUT /identity/v3/projects/9bb8362c5ac74c51b00fed8b524af56f/users/b5fec9a761cf4e18972eca72eff64152/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:10.878922 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4a300-cedb-4125-a21e-b8f94fb72b60 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.879585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4a300-cedb-4125-a21e-b8f94fb72b60 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.880081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4a300-cedb-4125-a21e-b8f94fb72b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.921901 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a9e4a300-cedb-4125-a21e-b8f94fb72b60 None None] MFA Rules not processed for user `b5fec9a761cf4e18972eca72eff64152`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:10.923671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a9e4a300-cedb-4125-a21e-b8f94fb72b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:10.964104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:10.965579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:10.965871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:10.966158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:10.968329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:10.971550 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:10.989062 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1304/2613] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:10.992448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.007015 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb1097-608a-4eea-aa4e-6153e0d48f15 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.009776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.010851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.011141 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.011419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] PATH_INFO: `/v3/projects/6d10592e4de346f4bed0148f8e0899cd/users/5a91d4ad190e463d8f5e4c9af51d393a/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.021791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1310/2614] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:11.026396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4a300-cedb-4125-a21e-b8f94fb72b60 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.032233 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d10592e4de346f4bed0148f8e0899cd, user_id=5a91d4ad190e463d8f5e4c9af51d393a, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.034131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.038588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4a300-cedb-4125-a21e-b8f94fb72b60 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.044216 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.045075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.045432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.045785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.046621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.048735 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.056491 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1305/2615] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:10 2022] PUT /identity/v3/projects/6d10592e4de346f4bed0148f8e0899cd/users/5a91d4ad190e463d8f5e4c9af51d393a/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:11.058301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.060229 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:11.078204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1311/2616] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:11 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:11.082565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.106399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:11.127988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b5bb1-a1a2-4c6a-b599-805569bc5f6a admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.153428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954ddda-09ca-4fad-84b2-b69a84a9036a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.158862 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.159534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.159816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.160092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.160778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.163277 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.183321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1312/2617] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:11 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:11.187148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.188201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.188904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.189191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.189466 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.190192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.191784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.197015 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127785c7-89ba-4e38-8b33-1f734d5bd0ca admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.201791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.205155 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1306/2618] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:11.205295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.205295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.205295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.205295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.205295 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.206195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.216522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07606-94a6-4720-9d6d-df8047fd3f47 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.216712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1313/2619] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:11.220408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.221150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.222023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.222306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.222581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] PATH_INFO: `/v3/projects/6d10592e4de346f4bed0148f8e0899cd/users/5a91d4ad190e463d8f5e4c9af51d393a/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.225482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d10592e4de346f4bed0148f8e0899cd, user_id=5a91d4ad190e463d8f5e4c9af51d393a, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.228180 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.230566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1777e43-4ffb-4b44-a1d0-966dd27615f0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.235369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.236240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.236513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.236792 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] PATH_INFO: `/v3/projects/72912e61abce48b6a0ee27f5aac00732/users/9d62930f27f84bb999350d343e216607/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.238757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1307/2620] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:11 2022] PUT /identity/v3/projects/6d10592e4de346f4bed0148f8e0899cd/users/5a91d4ad190e463d8f5e4c9af51d393a/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:11.255042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72912e61abce48b6a0ee27f5aac00732, user_id=9d62930f27f84bb999350d343e216607, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.258308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.273964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1314/2621] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:11 2022] PUT /identity/v3/projects/72912e61abce48b6a0ee27f5aac00732/users/9d62930f27f84bb999350d343e216607/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:11.278067 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.347405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef647624-43d2-4f4e-bc90-b7ba01b1033d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.410548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64797ceb-ca6b-4b4d-9fa3-ef08d942b915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.411326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797ceb-ca6b-4b4d-9fa3-ef08d942b915 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.411714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797ceb-ca6b-4b4d-9fa3-ef08d942b915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.412132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797ceb-ca6b-4b4d-9fa3-ef08d942b915 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.412889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64797ceb-ca6b-4b4d-9fa3-ef08d942b915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.414485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64797ceb-ca6b-4b4d-9fa3-ef08d942b915 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.425763 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1308/2622] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:11.430269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.440869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f2acd-24a3-4fe0-82de-3a0bed465197 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.446105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.446993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.447306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.447587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] PATH_INFO: `/v3/projects/72912e61abce48b6a0ee27f5aac00732/users/9d62930f27f84bb999350d343e216607/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.449662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=72912e61abce48b6a0ee27f5aac00732, user_id=9d62930f27f84bb999350d343e216607, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.452695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.464486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1315/2623] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:11 2022] PUT /identity/v3/projects/72912e61abce48b6a0ee27f5aac00732/users/9d62930f27f84bb999350d343e216607/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:11.475578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea57c-b412-4161-a1f7-dd6b81f63ee7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.475853 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea57c-b412-4161-a1f7-dd6b81f63ee7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.476133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ea57c-b412-4161-a1f7-dd6b81f63ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.500207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:11.504477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-f91ea57c-b412-4161-a1f7-dd6b81f63ee7 None None] MFA Rules not processed for user `9d62930f27f84bb999350d343e216607`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:11.507310 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f91ea57c-b412-4161-a1f7-dd6b81f63ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:11.589496 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1309/2624] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:11.603687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.605645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:11.673164 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be361ac5-cb00-427b-b6ea-f7af89cb45ce admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.677752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7be60-f26f-497d-b462-9e322721b515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.678410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7be60-f26f-497d-b462-9e322721b515 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.678697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7be60-f26f-497d-b462-9e322721b515 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.678973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7be60-f26f-497d-b462-9e322721b515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.689104 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7be60-f26f-497d-b462-9e322721b515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.690559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7be60-f26f-497d-b462-9e322721b515 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.700857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1316/2625] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:11 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:11.775183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ea57c-b412-4161-a1f7-dd6b81f63ee7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:11.785812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ea57c-b412-4161-a1f7-dd6b81f63ee7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:11.789435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:11.790081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.790367 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.790644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.855606 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:11.857142 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:11.867134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1310/2626] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:11.907363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:11.907363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:11.907631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:11.935429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] MFA Rules not processed for user `5a91d4ad190e463d8f5e4c9af51d393a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:11.936651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:11.967042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:12.033102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1317/2627] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:12.334332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:12.336027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:12.340219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:12.349488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:12.350047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ef04-afae-46d5-972a-1117e31be90f service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:12.354438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:12.355096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:12.355381 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:12.355658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:12.357777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:12.416513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f9c41-c49b-4a9a-a7e8-d34fb7f0e1ac None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:12.420359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:12.421170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:12.421535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:12.421998 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:12.423872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:12.424318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:12.425862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:12.439597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1311/2628] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:12.499915 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:12.502175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:12.513609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1318/2629] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:13.651603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:13.669844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab101c9-2075-4797-be22-a4a110fd6fbe service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:13.674777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:13.675893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:13.676178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:13.676456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] PATH_INFO: `/v3/users/b6248d87f0504e8f9fe5f1269ac1cef3` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:13.678258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6248d87f0504e8f9fe5f1269ac1cef3)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:13.680245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:13.716102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:13.731540 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6248d87f0504e8f9fe5f1269ac1cef3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:13.732316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6248d87f0504e8f9fe5f1269ac1cef3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:13.739531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6248d87f0504e8f9fe5f1269ac1cef3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:13.741116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6248d87f0504e8f9fe5f1269ac1cef3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:13.748771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6248d87f0504e8f9fe5f1269ac1cef3'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:13.766870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1312/2630] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:13 2022] DELETE /identity/v3/users/b6248d87f0504e8f9fe5f1269ac1cef3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:14.198705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:14.210400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb8710-5226-474d-910e-91789a1a0288 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:14.278004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:14.278985 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:14.279273 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:14.279701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] PATH_INFO: `/v3/projects/c86fdfb15b0c4925847fea258a5cc532` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:14.281085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86fdfb15b0c4925847fea258a5cc532)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:14.284137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:14.305712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:14.347844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86fdfb15b0c4925847fea258a5cc532', 'request_id': 'req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.355482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86fdfb15b0c4925847fea258a5cc532', 'request_id': 'req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.375820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86fdfb15b0c4925847fea258a5cc532', 'request_id': 'req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.399294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] The token cache is being invalidate because project c86fdfb15b0c4925847fea258a5cc532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:14.399294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.399294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.400723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1319/2631] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:14 2022] DELETE /identity/v3/projects/c86fdfb15b0c4925847fea258a5cc532 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:14.405035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:14.406995 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:14.426061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1085c4-3093-4144-8175-de20c4fdf177 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:14.507705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:14.508901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:14.509421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:14.509703 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] PATH_INFO: `/v3/users/5a91d4ad190e463d8f5e4c9af51d393a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:14.511117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a91d4ad190e463d8f5e4c9af51d393a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:14.513087 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:14.574083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a91d4ad190e463d8f5e4c9af51d393a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.574862 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a91d4ad190e463d8f5e4c9af51d393a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.586078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a91d4ad190e463d8f5e4c9af51d393a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.587277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a91d4ad190e463d8f5e4c9af51d393a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.596269 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a91d4ad190e463d8f5e4c9af51d393a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:14.635986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1313/2632] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:14 2022] DELETE /identity/v3/users/5a91d4ad190e463d8f5e4c9af51d393a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:14.905140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:14.906597 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:14.926438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a97fea-931a-4c9d-96b3-6c7c90fe3b1e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:14.954254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:14.956232 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:15.014576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:15.080798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79158-d353-42ae-8e07-bc0573b18556 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.085550 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.086463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.087956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.088747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.089066 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.089179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] PATH_INFO: `/v3/users/deda392de1dc48abb29fab0e06309313` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.089417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.089702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] PATH_INFO: `/v3/projects/6d10592e4de346f4bed0148f8e0899cd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.090635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deda392de1dc48abb29fab0e06309313)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.091347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d10592e4de346f4bed0148f8e0899cd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.093379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.096618 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.185723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deda392de1dc48abb29fab0e06309313'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.187180 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deda392de1dc48abb29fab0e06309313'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.188006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d10592e4de346f4bed0148f8e0899cd', 'request_id': 'req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.189077 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deda392de1dc48abb29fab0e06309313'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.200019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d10592e4de346f4bed0148f8e0899cd', 'request_id': 'req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.200381 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deda392de1dc48abb29fab0e06309313'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.214089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d10592e4de346f4bed0148f8e0899cd', 'request_id': 'req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.221640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-fec15db6-2cf9-498d-bd5c-df08a86f9ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deda392de1dc48abb29fab0e06309313'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.238469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] The token cache is being invalidate because project 6d10592e4de346f4bed0148f8e0899cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:15.240238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.241950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f6f214df-f6ae-4f1b-92a1-5b3e43dd2daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:15.243117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1320/2633] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:14 2022] DELETE /identity/v3/users/deda392de1dc48abb29fab0e06309313 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:15.244671 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab236-a834-4217-a2ed-8acd44e98746 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.244996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab236-a834-4217-a2ed-8acd44e98746 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.245280 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab236-a834-4217-a2ed-8acd44e98746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.246681 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1314/2634] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:14 2022] DELETE /identity/v3/projects/6d10592e4de346f4bed0148f8e0899cd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:15.253807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.254101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.254382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.272888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c73ab236-a834-4217-a2ed-8acd44e98746 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:15.274135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c73ab236-a834-4217-a2ed-8acd44e98746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:15.277533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:15.278822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:15.291152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:15.359100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1321/2635] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:15.368535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1315/2636] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:15.375239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.376261 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dfa33-7e73-4de7-b857-eb4db3565fd3 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.376914 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:15.377511 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dfa33-7e73-4de7-b857-eb4db3565fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.378259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dfa33-7e73-4de7-b857-eb4db3565fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.403835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-167dfa33-7e73-4de7-b857-eb4db3565fd3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:15.405086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-167dfa33-7e73-4de7-b857-eb4db3565fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:15.416002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1322/2637] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:15.420465 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dfa33-7e73-4de7-b857-eb4db3565fd3 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.430717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dfa33-7e73-4de7-b857-eb4db3565fd3 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.435687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.436541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.437045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.437488 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.438218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:15.438530 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.440140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.454853 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1323/2638] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:15.459712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.465617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccbd1d-4e19-4404-adcf-5542b02c88a5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.471594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbfad-7d5a-4b91-b4d5-c577265c68ff admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.471742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.472430 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.473343 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.473821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.482416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.484437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.484731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.485043 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.485755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.487881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.490047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.491489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.497785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:15.514199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1316/2639] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:15.548543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1324/2640] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:15 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:15.549347 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.579628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f4748-c820-4626-99c5-b7a3d7672f0b service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.689705 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.690640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.691012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.691361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.692197 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.694417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.726174 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1317/2641] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:15 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:15.729936 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.741275 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5d5b-51ff-4d29-9963-81610fce98ee admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.746834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.747484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.747772 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.748051 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.748820 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.750467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.762225 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1325/2642] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:15.766200 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.779781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a1cb0-2bbc-4498-bee0-067bf94274d7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.785605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9d010-d3a1-43d5-8207-da427d81cc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.786840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9d010-d3a1-43d5-8207-da427d81cc2d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.787126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9d010-d3a1-43d5-8207-da427d81cc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.787563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9d010-d3a1-43d5-8207-da427d81cc2d admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.788636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9d010-d3a1-43d5-8207-da427d81cc2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.789985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9d010-d3a1-43d5-8207-da427d81cc2d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.802156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1318/2643] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:15.806683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.818924 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f6ec1-b2ee-4f1e-8021-dfc09044a0b5 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:15.824114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:15.825086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.825376 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.825656 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] PATH_INFO: `/v3/projects/f72b4afc6e804a92997631ea0b7e3621/users/39cfa0b06d95462281d1592406d6006f/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.844544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f72b4afc6e804a92997631ea0b7e3621, user_id=39cfa0b06d95462281d1592406d6006f, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:15.847283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:15.863688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1326/2644] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:15 2022] PUT /identity/v3/projects/f72b4afc6e804a92997631ea0b7e3621/users/39cfa0b06d95462281d1592406d6006f/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:15.874357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef89c14-2ed9-4f95-ad0b-1090596cda72 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:15.874675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef89c14-2ed9-4f95-ad0b-1090596cda72 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:15.875490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef89c14-2ed9-4f95-ad0b-1090596cda72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:15.926103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:15.928391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:15.934457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-aef89c14-2ed9-4f95-ad0b-1090596cda72 None None] MFA Rules not processed for user `39cfa0b06d95462281d1592406d6006f`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:15.936033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-aef89c14-2ed9-4f95-ad0b-1090596cda72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:16.038659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1319/2645] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:16.043959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef89c14-2ed9-4f95-ad0b-1090596cda72 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.070035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd75eb4-0bfc-443d-b74e-867bd7c34503 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.075411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.076307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.076574 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.076875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] PATH_INFO: `/v3/projects/6e8557bf30564fe09f4ed7ff8efd241f` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.079731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8557bf30564fe09f4ed7ff8efd241f)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.081724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.137033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef89c14-2ed9-4f95-ad0b-1090596cda72 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.146971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:16.152463 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:16.152627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8557bf30564fe09f4ed7ff8efd241f', 'request_id': 'req-07d001ff-d24e-4031-a3d5-e74d4521c01c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.166255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8557bf30564fe09f4ed7ff8efd241f', 'request_id': 'req-07d001ff-d24e-4031-a3d5-e74d4521c01c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.179810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8557bf30564fe09f4ed7ff8efd241f', 'request_id': 'req-07d001ff-d24e-4031-a3d5-e74d4521c01c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.195187 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] The token cache is being invalidate because project 6e8557bf30564fe09f4ed7ff8efd241f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:16.195525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.196244 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.198322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1327/2646] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:15 2022] DELETE /identity/v3/projects/6e8557bf30564fe09f4ed7ff8efd241f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:16.203086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.204932 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:16.214220 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.215589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.215878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.216154 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.216878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.219270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.228075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d001ff-d24e-4031-a3d5-e74d4521c01c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.233313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.234200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.234485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.235057 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] PATH_INFO: `/v3/users/ded5e005037f400aa9c29eaefdd94769` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.236326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ded5e005037f400aa9c29eaefdd94769)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.238343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.255532 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1320/2647] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:16 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:16.257289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.264546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:16.330959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ded5e005037f400aa9c29eaefdd94769'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.332404 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ded5e005037f400aa9c29eaefdd94769'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.333144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ded5e005037f400aa9c29eaefdd94769'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.348599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ded5e005037f400aa9c29eaefdd94769'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.374939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ded5e005037f400aa9c29eaefdd94769'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:16.381126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200a742-a261-4203-a9ab-4d067594ee16 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.390941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.392017 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.392790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.393329 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.394288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.399634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1328/2648] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:16 2022] DELETE /identity/v3/users/ded5e005037f400aa9c29eaefdd94769 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:16.402133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.435762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.437383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:16.440348 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1321/2649] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:16 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:16.540943 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fec687-82c4-4c5c-ab13-ad7d5dcff32a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.546313 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.546997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.547417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.547556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.548518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.550566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.563310 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1329/2650] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:16.568160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.578554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e0698-e820-4e70-b482-2fea59cd43e0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.584336 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.585493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.585776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.586049 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] PATH_INFO: `/v3/projects/d7ba0b35485d437796d7ead291ea5c9d/users/2a6d4404b8c54f22ad365e3df7a30e2e/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.605283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ba0b35485d437796d7ead291ea5c9d, user_id=2a6d4404b8c54f22ad365e3df7a30e2e, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.608178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.633933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1322/2651] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:16 2022] PUT /identity/v3/projects/d7ba0b35485d437796d7ead291ea5c9d/users/2a6d4404b8c54f22ad365e3df7a30e2e/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:16.637953 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.649403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d81d0-4062-4bf6-8936-6511b983f488 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.701234 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:16.734388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0dd09-baab-4a5b-b8bc-9b24646dbeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.735452 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0dd09-baab-4a5b-b8bc-9b24646dbeff admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.735805 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0dd09-baab-4a5b-b8bc-9b24646dbeff admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.736162 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0dd09-baab-4a5b-b8bc-9b24646dbeff admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.737294 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0dd09-baab-4a5b-b8bc-9b24646dbeff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.739387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0dd09-baab-4a5b-b8bc-9b24646dbeff admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.750872 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1330/2652] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:16.756305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.767479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1eda67-c47d-4b8c-95f3-e1c729cf4a86 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.773121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.774091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.774287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.774563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] PATH_INFO: `/v3/projects/d7ba0b35485d437796d7ead291ea5c9d/users/2a6d4404b8c54f22ad365e3df7a30e2e/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.776592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ba0b35485d437796d7ead291ea5c9d, user_id=2a6d4404b8c54f22ad365e3df7a30e2e, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.779767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.795523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1323/2653] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:16 2022] PUT /identity/v3/projects/d7ba0b35485d437796d7ead291ea5c9d/users/2a6d4404b8c54f22ad365e3df7a30e2e/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:16.807767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.813142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.813462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.837700 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.841476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:16.851203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] MFA Rules not processed for user `2a6d4404b8c54f22ad365e3df7a30e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:16.852427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:16.925005 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:16.939207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1331/2654] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:16.940877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839fc2d-c1f9-487d-a991-1ac59ff278d3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:16.947859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:16.948737 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:16.949043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:16.949324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] PATH_INFO: `/v3/projects/2580748d5071406098e2f1710e4eab93` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:16.951159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2580748d5071406098e2f1710e4eab93)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:16.953007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:16.953143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:16.954849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:17.006973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2580748d5071406098e2f1710e4eab93', 'request_id': 'req-11ca7dac-e57c-4ce3-b7da-1b8273720643'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.013749 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2580748d5071406098e2f1710e4eab93', 'request_id': 'req-11ca7dac-e57c-4ce3-b7da-1b8273720643'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.022909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2580748d5071406098e2f1710e4eab93', 'request_id': 'req-11ca7dac-e57c-4ce3-b7da-1b8273720643'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.023096 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749189e-f967-4da2-89be-2e8319b47bab None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:17.036413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] The token cache is being invalidate because project 2580748d5071406098e2f1710e4eab93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:17.036780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.037589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.039607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1324/2655] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:16 2022] DELETE /identity/v3/projects/2580748d5071406098e2f1710e4eab93 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:17.043098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:17.044518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:17.077713 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:17.078340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:17.078559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:17.078825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:17.081027 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:17.123291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:17.127810 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca7dac-e57c-4ce3-b7da-1b8273720643 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:17.132788 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:17.133707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:17.133988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:17.134435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] PATH_INFO: `/v3/users/e839fc95db0e4bc29fa3eeeb3c013a0a` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:17.135680 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e839fc95db0e4bc29fa3eeeb3c013a0a)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:17.137682 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:17.166697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:17.168393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:17.182064 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1332/2656] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:17.193807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e839fc95db0e4bc29fa3eeeb3c013a0a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.194743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e839fc95db0e4bc29fa3eeeb3c013a0a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.202487 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e839fc95db0e4bc29fa3eeeb3c013a0a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.203552 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e839fc95db0e4bc29fa3eeeb3c013a0a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.212234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a54495e3-0d82-44fc-b1d1-e475bfe9e5fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e839fc95db0e4bc29fa3eeeb3c013a0a'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.235136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1325/2657] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:17 2022] DELETE /identity/v3/users/e839fc95db0e4bc29fa3eeeb3c013a0a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:17.547482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:17.563256 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115850-6b18-4757-90ad-cb92fbb3ec36 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:17.652413 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:17.653424 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:17.653912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:17.654322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] PATH_INFO: `/v3/projects/72919baf4182444996dbf3920b53479d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:17.655628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72919baf4182444996dbf3920b53479d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:17.657701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:17.711102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72919baf4182444996dbf3920b53479d', 'request_id': 'req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.719318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72919baf4182444996dbf3920b53479d', 'request_id': 'req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.727826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72919baf4182444996dbf3920b53479d', 'request_id': 'req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.742485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] The token cache is being invalidate because project 72919baf4182444996dbf3920b53479d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:17.742645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.743461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7d2d5ad7-b6f9-42e9-9c88-90227dedef98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:17.749978 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1333/2658] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:17 2022] DELETE /identity/v3/projects/72919baf4182444996dbf3920b53479d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:17.752203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:17.753041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:17.753493 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:17.773988 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:17.791779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:17.793774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:17.909161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1326/2659] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:17.917523 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc5b0b-ba73-4d95-a81a-99d16f1528db None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:17.917812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc5b0b-ba73-4d95-a81a-99d16f1528db None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:17.918097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc5b0b-ba73-4d95-a81a-99d16f1528db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:17.920009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:17.921753 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:17.935770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3190b57-dd12-4bbe-a335-bf2c7d4ad08d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:17.940560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-3fdc5b0b-ba73-4d95-a81a-99d16f1528db None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:17.942334 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3fdc5b0b-ba73-4d95-a81a-99d16f1528db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:17.943511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:17.944451 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:17.944742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:17.945621 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] PATH_INFO: `/v3/users/39cfa0b06d95462281d1592406d6006f` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:17.948660 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39cfa0b06d95462281d1592406d6006f)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:17.950628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:17.952152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1334/2660] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:17.956165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc5b0b-ba73-4d95-a81a-99d16f1528db None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:17.967973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc5b0b-ba73-4d95-a81a-99d16f1528db None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:17.975197 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:17.975879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:17.976163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:17.976440 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:17.977220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:17.977534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:17.979397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:17.997355 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1335/2661] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:17.998968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.012170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070eae5-4cf3-4027-a558-913792a01638 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.014425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39cfa0b06d95462281d1592406d6006f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.015291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39cfa0b06d95462281d1592406d6006f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.019361 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.020321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.021770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.021770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.021770 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.023191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39cfa0b06d95462281d1592406d6006f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.023568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.024104 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39cfa0b06d95462281d1592406d6006f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.033567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-e271111a-fca8-4cbe-954e-1c1cd26c043f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39cfa0b06d95462281d1592406d6006f'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.056668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1336/2662] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:17 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:18.063001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1327/2663] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:17 2022] DELETE /identity/v3/users/39cfa0b06d95462281d1592406d6006f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:18.063543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.064716 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.065046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.065388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:18.065508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.084038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:18.085353 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:18.089468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:18.142443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b85aac-e340-4644-9d2b-eed826337a05 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.154535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1328/2664] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:18.165974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.168145 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:18.191472 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:18.213345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.214028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.214315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.214594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.215286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.217397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.238844 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f4ad4-4bf9-4f16-ac72-7feca41c0379 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.243183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.243838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.244116 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.244389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.245343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1337/2665] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:18 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:18.249265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.255109 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.256631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.261478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74236c54-920f-4fbd-8f64-7163073b4bf4 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.266611 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1329/2666] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:18.269518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.270190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.271115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.271410 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.272253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.273892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.287447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1338/2667] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:18.291802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.306698 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afa1de-6424-40a3-a7b2-87fc3a10ec8a service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.310891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6117e46-7ed7-4f84-83f3-71c984673a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.311584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6117e46-7ed7-4f84-83f3-71c984673a78 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.311870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6117e46-7ed7-4f84-83f3-71c984673a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.312246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6117e46-7ed7-4f84-83f3-71c984673a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.312898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6117e46-7ed7-4f84-83f3-71c984673a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.314505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6117e46-7ed7-4f84-83f3-71c984673a78 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.329057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1330/2668] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:18.330111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.343050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409ddd2-773f-4208-ab51-5e079b065659 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.349699 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.350960 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.351419 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.352031 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] PATH_INFO: `/v3/projects/08daf899a87647ce882bd9b3d344e733/users/0231c20c33154cb8903708b84aed9739/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.374646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=08daf899a87647ce882bd9b3d344e733, user_id=0231c20c33154cb8903708b84aed9739, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.378476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.398074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1339/2669] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:18 2022] PUT /identity/v3/projects/08daf899a87647ce882bd9b3d344e733/users/0231c20c33154cb8903708b84aed9739/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:18.409583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f94a-64d5-492e-a9d7-dc0436ac808b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.409899 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f94a-64d5-492e-a9d7-dc0436ac808b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.410146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f94a-64d5-492e-a9d7-dc0436ac808b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.442150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b2e0f94a-64d5-492e-a9d7-dc0436ac808b None None] MFA Rules not processed for user `0231c20c33154cb8903708b84aed9739`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:18.443402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b2e0f94a-64d5-492e-a9d7-dc0436ac808b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:18.525257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1331/2670] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:18.529637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.605137 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a5344-de1b-4f92-8eea-2497ed3cc90c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.610673 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.611991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.612485 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.612997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.613911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.616134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.669409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1340/2671] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:18 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:18.673712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f94a-64d5-492e-a9d7-dc0436ac808b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.686115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f94a-64d5-492e-a9d7-dc0436ac808b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.764891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.765640 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.765929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.766209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.766919 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.768858 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.794505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1332/2672] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:18 2022] POST /identity/v3/users => generated 298 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:18.798547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.806752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:18.811268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.819960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:18.820392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da06dd0-c985-4848-8dde-6136ed462f28 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.826784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.827472 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.827867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.828165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.829727 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.831124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.839872 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5e853-ee56-40c2-89ec-4fbbeb9a8dc0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.846906 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1341/2673] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:18.850855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.851011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.852081 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.852519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.852818 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] PATH_INFO: `/v3/projects/f72b4afc6e804a92997631ea0b7e3621` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.854099 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f72b4afc6e804a92997631ea0b7e3621)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.856068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.865656 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f99c0d-2e7e-4683-976e-fd54f884c184 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:18.871288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:18.872429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:18.872949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:18.873332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] PATH_INFO: `/v3/projects/8eb8ee24c90542fe9c96c1e099561c24/users/e1d0873baaec404fa087b920732f947c/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:18.896603 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:18.902811 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb8ee24c90542fe9c96c1e099561c24, user_id=e1d0873baaec404fa087b920732f947c, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:18.906640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:18.926638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1342/2674] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:18 2022] PUT /identity/v3/projects/8eb8ee24c90542fe9c96c1e099561c24/users/e1d0873baaec404fa087b920732f947c/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:18.934235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.940910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f72b4afc6e804a92997631ea0b7e3621', 'request_id': 'req-6bbf155a-2734-421e-939f-1c2909e0263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.953075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f72b4afc6e804a92997631ea0b7e3621', 'request_id': 'req-6bbf155a-2734-421e-939f-1c2909e0263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.969209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f72b4afc6e804a92997631ea0b7e3621', 'request_id': 'req-6bbf155a-2734-421e-939f-1c2909e0263c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.986707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] The token cache is being invalidate because project f72b4afc6e804a92997631ea0b7e3621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:18.987261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.988251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:18.992048 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1333/2675] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:18 2022] DELETE /identity/v3/projects/f72b4afc6e804a92997631ea0b7e3621 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:18.996238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:18.998407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:19.027504 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3847dc-e3ac-4de2-8951-fa9b202d1de0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:19.087652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf155a-2734-421e-939f-1c2909e0263c admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:19.093770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:19.094847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:19.095161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:19.095467 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] PATH_INFO: `/v3/users/2a6d4404b8c54f22ad365e3df7a30e2e` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:19.097469 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6d4404b8c54f22ad365e3df7a30e2e)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:19.100072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:19.116260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:19.116877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:19.117170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:19.117509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:19.118251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:19.120577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:19.136759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1343/2676] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:19.139543 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:19.142010 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:19.159245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dcec3-2a01-4d36-93d4-d9a3445afe60 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:19.165425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6d4404b8c54f22ad365e3df7a30e2e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:19.165641 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a86803-5b58-4828-8978-40797075c8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:19.166420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6d4404b8c54f22ad365e3df7a30e2e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:19.166560 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86803-5b58-4828-8978-40797075c8cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:19.166814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86803-5b58-4828-8978-40797075c8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:19.167099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a86803-5b58-4828-8978-40797075c8cc admin admin] PATH_INFO: `/v3/projects/8eb8ee24c90542fe9c96c1e099561c24/users/e1d0873baaec404fa087b920732f947c/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:19.169715 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a86803-5b58-4828-8978-40797075c8cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb8ee24c90542fe9c96c1e099561c24, user_id=e1d0873baaec404fa087b920732f947c, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:19.172557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a86803-5b58-4828-8978-40797075c8cc admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:19.174283 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6d4404b8c54f22ad365e3df7a30e2e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:19.175075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6d4404b8c54f22ad365e3df7a30e2e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:19.184937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6d4404b8c54f22ad365e3df7a30e2e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:19.186679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1344/2677] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:19 2022] PUT /identity/v3/projects/8eb8ee24c90542fe9c96c1e099561c24/users/e1d0873baaec404fa087b920732f947c/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:19.198395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baea8d-b99f-4408-a39f-3e75bc9cda36 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:19.198769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baea8d-b99f-4408-a39f-3e75bc9cda36 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:19.199117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78baea8d-b99f-4408-a39f-3e75bc9cda36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:19.215432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1334/2678] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:18 2022] DELETE /identity/v3/users/2a6d4404b8c54f22ad365e3df7a30e2e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:19.234672 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-78baea8d-b99f-4408-a39f-3e75bc9cda36 None None] MFA Rules not processed for user `e1d0873baaec404fa087b920732f947c`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:19.235949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-78baea8d-b99f-4408-a39f-3e75bc9cda36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:19.291757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:19.318143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1345/2679] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:19.330293 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:19.331579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:19.384523 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:19.443553 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfde48-189d-4364-9b7f-39d04c174f01 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:19.448043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:19.448807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:19.449366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:19.449735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:19.462120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:19.463877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:19.482389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1335/2680] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:19.604378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78baea8d-b99f-4408-a39f-3e75bc9cda36 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:19.615382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78baea8d-b99f-4408-a39f-3e75bc9cda36 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:19.620605 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8148e5-56eb-40a4-8a2c-5d145688ae2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:19.621315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8148e5-56eb-40a4-8a2c-5d145688ae2b service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:19.621668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8148e5-56eb-40a4-8a2c-5d145688ae2b service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:19.622013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8148e5-56eb-40a4-8a2c-5d145688ae2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:19.623878 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-cb8148e5-56eb-40a4-8a2c-5d145688ae2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:19.686170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8148e5-56eb-40a4-8a2c-5d145688ae2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:19.687939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8148e5-56eb-40a4-8a2c-5d145688ae2b service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:19.698639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1346/2681] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:19.718998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:19.720475 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:19.823033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e53fe-2bd2-4105-9c28-8c42a9f58f04 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:19.828086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:19.829150 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:19.829435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:19.829704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] PATH_INFO: `/v3/projects/d7ba0b35485d437796d7ead291ea5c9d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:19.831300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ba0b35485d437796d7ead291ea5c9d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:19.833847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:19.937666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ba0b35485d437796d7ead291ea5c9d', 'request_id': 'req-c4a56895-6a37-41af-b32c-7958f0f863f0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:19.964362 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ba0b35485d437796d7ead291ea5c9d', 'request_id': 'req-c4a56895-6a37-41af-b32c-7958f0f863f0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:19.983809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ba0b35485d437796d7ead291ea5c9d', 'request_id': 'req-c4a56895-6a37-41af-b32c-7958f0f863f0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.013424 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] The token cache is being invalidate because project d7ba0b35485d437796d7ead291ea5c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:20.013726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.014662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c4a56895-6a37-41af-b32c-7958f0f863f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.016401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1336/2682] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:19 2022] DELETE /identity/v3/projects/d7ba0b35485d437796d7ead291ea5c9d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:20.021937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7c12f-7876-49b0-b3e3-f462d18447b6 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.022596 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7c12f-7876-49b0-b3e3-f462d18447b6 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.023884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7c12f-7876-49b0-b3e3-f462d18447b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.064609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-00b7c12f-7876-49b0-b3e3-f462d18447b6 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:20.070788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-00b7c12f-7876-49b0-b3e3-f462d18447b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:20.198720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1347/2683] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:20.209212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de8e55-9b86-4357-8cce-4acd8c6a0594 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.210446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de8e55-9b86-4357-8cce-4acd8c6a0594 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.211034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de8e55-9b86-4357-8cce-4acd8c6a0594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.260697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-93de8e55-9b86-4357-8cce-4acd8c6a0594 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:20.263419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-93de8e55-9b86-4357-8cce-4acd8c6a0594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:20.284572 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1337/2684] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:20.286091 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7c12f-7876-49b0-b3e3-f462d18447b6 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.297928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7c12f-7876-49b0-b3e3-f462d18447b6 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.304631 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.305344 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.305685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.306060 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.307161 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:20.307492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.309176 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.323332 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1348/2685] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:20.327357 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de8e55-9b86-4357-8cce-4acd8c6a0594 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.341068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de8e55-9b86-4357-8cce-4acd8c6a0594 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.346506 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.347192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.347481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.348058 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.348508 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.350699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.385091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1338/2686] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:20 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:20.389453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.406669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2594b6c-50ec-4111-8cea-fcbd63804cdb admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.475646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.476451 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.476771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.477105 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.477839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.479825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.496331 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:20.508106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1349/2687] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:20 2022] POST /identity/v3/users => generated 282 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:20.509747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.520443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2995660-1fcc-4d00-a208-1ea54b7c80a3 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.526195 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.526873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.527158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.527436 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.528177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.529929 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.545034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1339/2688] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:20.546201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.557194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14d85e-9526-4883-bdda-3313fc054e37 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.562039 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291ec7a-95a0-47b8-bc37-87b0e655cd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.562721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ec7a-95a0-47b8-bc37-87b0e655cd26 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.563007 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ec7a-95a0-47b8-bc37-87b0e655cd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.563288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291ec7a-95a0-47b8-bc37-87b0e655cd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.564028 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291ec7a-95a0-47b8-bc37-87b0e655cd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.565710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291ec7a-95a0-47b8-bc37-87b0e655cd26 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.577536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1350/2689] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:20.581942 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.592823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13217dea-237c-448a-9072-3c58d643ec4d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.597903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.598840 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.599125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.599402 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] PATH_INFO: `/v3/projects/784e4d4b1a4d4bc9ac4f016a57ec2cfd/users/9333d2fc707140cf99870047c9011eb3/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.616098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=784e4d4b1a4d4bc9ac4f016a57ec2cfd, user_id=9333d2fc707140cf99870047c9011eb3, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.618794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.628207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:20.635555 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1340/2690] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:20 2022] PUT /identity/v3/projects/784e4d4b1a4d4bc9ac4f016a57ec2cfd/users/9333d2fc707140cf99870047c9011eb3/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:20.646263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.646800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.646800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.676591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] MFA Rules not processed for user `9333d2fc707140cf99870047c9011eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:20.678374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:20.747477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1351/2691] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:20.758888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.760783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:20.802720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:20.804179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:20.832940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59641efc-9282-4b1f-be79-091b4aeabebe admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.837213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.838028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.838312 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.838586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.847117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.848751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.858460 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1341/2692] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:20 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:20.877508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7316-863f-49ce-b63b-1d4b2bc73c42 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:20.882276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:20.883165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:20.883443 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:20.883728 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] PATH_INFO: `/v3/users/9d977f609537476daa8fe8adbab9ff9c` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:20.898205 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d977f609537476daa8fe8adbab9ff9c)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:20.900089 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:20.947479 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d977f609537476daa8fe8adbab9ff9c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.948154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d977f609537476daa8fe8adbab9ff9c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.948785 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d977f609537476daa8fe8adbab9ff9c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.955795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d977f609537476daa8fe8adbab9ff9c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.966651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:20.969541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d977f609537476daa8fe8adbab9ff9c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:20.983663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1352/2693] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:20 2022] DELETE /identity/v3/users/9d977f609537476daa8fe8adbab9ff9c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:21.192242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:21.206673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a9781-0535-473e-a095-dbcbb616659d service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:21.299932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:21.300809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:21.301125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:21.301403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] PATH_INFO: `/v3/projects/05223500b4a142c7a77c2de010541b84` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:21.312501 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05223500b4a142c7a77c2de010541b84)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:21.315943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:21.368880 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05223500b4a142c7a77c2de010541b84', 'request_id': 'req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.374781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05223500b4a142c7a77c2de010541b84', 'request_id': 'req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.383408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05223500b4a142c7a77c2de010541b84', 'request_id': 'req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.395044 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] The token cache is being invalidate because project 05223500b4a142c7a77c2de010541b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:21.395341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.396023 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.397856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1342/2694] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:21 2022] DELETE /identity/v3/projects/05223500b4a142c7a77c2de010541b84 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:21.401689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:21.403224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:21.417356 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78edb04-e385-44f2-b876-c5a4509313e3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:21.488302 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:21.490877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:21.491175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:21.491522 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] PATH_INFO: `/v3/users/4b3012ed20354085ac032f78a2fa91b8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:21.508086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b3012ed20354085ac032f78a2fa91b8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:21.510093 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:21.561762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b3012ed20354085ac032f78a2fa91b8'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.562412 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b3012ed20354085ac032f78a2fa91b8'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.563025 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b3012ed20354085ac032f78a2fa91b8'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.570507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b3012ed20354085ac032f78a2fa91b8'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.582203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b3012ed20354085ac032f78a2fa91b8'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.594487 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1353/2695] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:21 2022] DELETE /identity/v3/users/4b3012ed20354085ac032f78a2fa91b8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:21.763911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:21.765403 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:21.778181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa8eb9-f4d8-4249-bfb0-0149fb7c38e1 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:21.798431 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:21.837817 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:21.838676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:21.838960 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:21.839238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] PATH_INFO: `/v3/projects/0531c85fd17e44fcbac42ff8e5496707` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:21.850009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0531c85fd17e44fcbac42ff8e5496707)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:21.851922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:21.898512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0531c85fd17e44fcbac42ff8e5496707', 'request_id': 'req-a7277671-a961-4256-bc17-bedf5279885e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.903924 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0531c85fd17e44fcbac42ff8e5496707', 'request_id': 'req-a7277671-a961-4256-bc17-bedf5279885e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.911766 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0531c85fd17e44fcbac42ff8e5496707', 'request_id': 'req-a7277671-a961-4256-bc17-bedf5279885e'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.923238 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] The token cache is being invalidate because project 0531c85fd17e44fcbac42ff8e5496707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:21.923524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.924153 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a7277671-a961-4256-bc17-bedf5279885e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:21.925724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1343/2696] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:21 2022] DELETE /identity/v3/projects/0531c85fd17e44fcbac42ff8e5496707 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:23.553517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:23.555026 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:23.567769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2fee0-af4a-4ca9-bae9-6e96fa370a72 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:23.618189 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:23.625905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:23.626845 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:23.627292 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:23.627778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] PATH_INFO: `/v3/users/0231c20c33154cb8903708b84aed9739` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:23.629134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0231c20c33154cb8903708b84aed9739)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:23.631683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:23.677925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0231c20c33154cb8903708b84aed9739'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:23.678510 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0231c20c33154cb8903708b84aed9739'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:23.679115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0231c20c33154cb8903708b84aed9739'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:23.686065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0231c20c33154cb8903708b84aed9739'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:23.698484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0231c20c33154cb8903708b84aed9739'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:23.710898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1354/2697] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:23 2022] DELETE /identity/v3/users/0231c20c33154cb8903708b84aed9739 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:23.715740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:23.716311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:23.716311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:23.733794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:23.735032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:23.799006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1344/2698] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:23.813236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:23.813236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:23.879802 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9056645-9706-4373-bb76-29ff59333265 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:23.884107 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d73515-91f0-4eb6-be0e-465d9865d39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:23.884959 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73515-91f0-4eb6-be0e-465d9865d39d service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:23.885095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73515-91f0-4eb6-be0e-465d9865d39d service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:23.885354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d73515-91f0-4eb6-be0e-465d9865d39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:23.895665 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d73515-91f0-4eb6-be0e-465d9865d39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:23.899585 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d73515-91f0-4eb6-be0e-465d9865d39d service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:23.908170 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1355/2699] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:24.039262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:24.040663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.052675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f67be-ec32-4a39-8613-ebfc219a7e6f None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:24.057551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:24.058478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:24.058785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:24.059033 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] PATH_INFO: `/v3/users/b5fec9a761cf4e18972eca72eff64152` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:24.060804 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5fec9a761cf4e18972eca72eff64152)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:24.062461 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:24.106322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:24.113179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5fec9a761cf4e18972eca72eff64152'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.114134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5fec9a761cf4e18972eca72eff64152'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.122863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5fec9a761cf4e18972eca72eff64152'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.123580 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5fec9a761cf4e18972eca72eff64152'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.130981 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5fec9a761cf4e18972eca72eff64152'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.152178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1345/2700] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:24 2022] DELETE /identity/v3/users/b5fec9a761cf4e18972eca72eff64152 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:24.156806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:24.157141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:24.157421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:24.175499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:24.176816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.206171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:24.207685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.255645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1356/2701] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:24.265041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:24.266476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.281112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85ae1f-1b64-491e-b1ad-d57941d5e349 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:24.286204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:24.287442 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:24.287599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:24.287838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:24.288433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] PATH_INFO: `/v3/projects/08daf899a87647ce882bd9b3d344e733` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:24.289500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08daf899a87647ce882bd9b3d344e733)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:24.291482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:24.326152 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22af785-4cca-429f-ab95-7ed23c65f76f None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:24.339271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08daf899a87647ce882bd9b3d344e733', 'request_id': 'req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.345032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08daf899a87647ce882bd9b3d344e733', 'request_id': 'req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.353476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08daf899a87647ce882bd9b3d344e733', 'request_id': 'req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.365679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] The token cache is being invalidate because project 08daf899a87647ce882bd9b3d344e733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:24.365974 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.366633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.368209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1346/2702] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:24 2022] DELETE /identity/v3/projects/08daf899a87647ce882bd9b3d344e733 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:24.372068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:24.373464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.378167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:24.378911 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:24.379284 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:24.379573 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:24.382213 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.426331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:24.444573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d7196-ba00-4e52-b9d8-e58b0c4d4bc7 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:24.449317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:24.450133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:24.450420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:24.450526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:24.450699 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] PATH_INFO: `/v3/users/e1d0873baaec404fa087b920732f947c` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:24.451880 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:24.452001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d0873baaec404fa087b920732f947c)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:24.453878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:24.461708 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1357/2703] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:10:24.502767 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d0873baaec404fa087b920732f947c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.503530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d0873baaec404fa087b920732f947c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.512597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d0873baaec404fa087b920732f947c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.515689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d0873baaec404fa087b920732f947c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.524082 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d0873baaec404fa087b920732f947c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.544525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1347/2704] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:24 2022] DELETE /identity/v3/users/e1d0873baaec404fa087b920732f947c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:24.815849 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:24.817411 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.830141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde67a-05ff-4260-8062-3244a723769d service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:24.899354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:24.900224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:24.900579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:24.900881 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] PATH_INFO: `/v3/projects/9bb8362c5ac74c51b00fed8b524af56f` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:24.902083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb8362c5ac74c51b00fed8b524af56f)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:24.904004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:24.922086 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:24.923694 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:24.937524 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:24.940938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a7fab-1ff6-48cc-8c1b-eeb2ed1dba37 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:24.948918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:24.951225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:24.951631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:24.952050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] PATH_INFO: `/v3/projects/8eb8ee24c90542fe9c96c1e099561c24` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:24.954237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb8ee24c90542fe9c96c1e099561c24)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:24.956738 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:24.961646 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb8362c5ac74c51b00fed8b524af56f', 'request_id': 'req-51425ece-f999-49b3-ae08-201f79474b92'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.968730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb8362c5ac74c51b00fed8b524af56f', 'request_id': 'req-51425ece-f999-49b3-ae08-201f79474b92'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.980683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb8362c5ac74c51b00fed8b524af56f', 'request_id': 'req-51425ece-f999-49b3-ae08-201f79474b92'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.995868 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] The token cache is being invalidate because project 9bb8362c5ac74c51b00fed8b524af56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:24.995967 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.996724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:24.998159 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1358/2705] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:24 2022] DELETE /identity/v3/projects/9bb8362c5ac74c51b00fed8b524af56f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:25.002337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.004393 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.018031 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb8ee24c90542fe9c96c1e099561c24', 'request_id': 'req-a1b8f2f1-5859-4827-9965-4e466215bd75'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.024778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb8ee24c90542fe9c96c1e099561c24', 'request_id': 'req-a1b8f2f1-5859-4827-9965-4e466215bd75'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.033262 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb8ee24c90542fe9c96c1e099561c24', 'request_id': 'req-a1b8f2f1-5859-4827-9965-4e466215bd75'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.046378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] The token cache is being invalidate because project 8eb8ee24c90542fe9c96c1e099561c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:25.046861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.047528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-a1b8f2f1-5859-4827-9965-4e466215bd75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.049171 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1348/2706] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:24 2022] DELETE /identity/v3/projects/8eb8ee24c90542fe9c96c1e099561c24 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:25.055379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d7af8-f36e-4cf8-9d06-c6baeb7ace57 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.055866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d7af8-f36e-4cf8-9d06-c6baeb7ace57 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.056160 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d7af8-f36e-4cf8-9d06-c6baeb7ace57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.083457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c61d7af8-f36e-4cf8-9d06-c6baeb7ace57 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:25.084747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c61d7af8-f36e-4cf8-9d06-c6baeb7ace57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.096598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51425ece-f999-49b3-ae08-201f79474b92 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.132158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:25.150175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1349/2707] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:25.154807 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.155757 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.156050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.156320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] PATH_INFO: `/v3/users/9d62930f27f84bb999350d343e216607` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.157544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d62930f27f84bb999350d343e216607)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.159260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be61058-7503-42a9-9bbd-9a7c7555bee3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.159598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be61058-7503-42a9-9bbd-9a7c7555bee3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.159717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.159882 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be61058-7503-42a9-9bbd-9a7c7555bee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.178629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0be61058-7503-42a9-9bbd-9a7c7555bee3 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:25.179852 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0be61058-7503-42a9-9bbd-9a7c7555bee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.188605 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1350/2708] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:25.192822 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be61058-7503-42a9-9bbd-9a7c7555bee3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.202619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be61058-7503-42a9-9bbd-9a7c7555bee3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.207855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.208567 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.208965 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.209319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.210178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:25.210588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.212295 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.214023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d62930f27f84bb999350d343e216607'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.214705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d62930f27f84bb999350d343e216607'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.215635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d62930f27f84bb999350d343e216607'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.223048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d62930f27f84bb999350d343e216607'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.225100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1351/2709] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:25.229228 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.229469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:25.231007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.238374 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d62930f27f84bb999350d343e216607'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.245344 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df0ebb-4e59-4785-984d-ccc0ed3b0608 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.250510 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.251165 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.251449 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.251724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.252454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.253086 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1359/2710] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:25 2022] DELETE /identity/v3/users/9d62930f27f84bb999350d343e216607 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:25.254500 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.284743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1352/2711] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:25.286547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.359812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6411035-bfe6-4b63-a6d6-04ac0dd53f5e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.421949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.422951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.423236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.423512 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.424201 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.426163 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.448329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1360/2712] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:25.452284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.463277 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a69a34-4454-4bbe-8346-4b0f929fb661 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.467616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.468278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.468560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.468978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.469707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.471311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.482088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1353/2713] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:25.486092 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.496123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4456ff1-4a01-41dd-b3fd-3aa8a76cd66e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.501085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.501758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.502045 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.502326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.503475 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.504693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.515576 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1361/2714] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:25.519693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.529937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f3ec1-82ce-48db-8355-592d3628561b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.535103 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b557db-672e-4295-a083-8f0f76439910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.535993 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b557db-672e-4295-a083-8f0f76439910 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.536305 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b557db-672e-4295-a083-8f0f76439910 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.536583 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b557db-672e-4295-a083-8f0f76439910 admin admin] PATH_INFO: `/v3/projects/13cbc80d2355452997c6d5e57b8bcc6b/users/dfd7c44ad6734801a470370321b1322c/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.553409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b557db-672e-4295-a083-8f0f76439910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13cbc80d2355452997c6d5e57b8bcc6b, user_id=dfd7c44ad6734801a470370321b1322c, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.556794 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b557db-672e-4295-a083-8f0f76439910 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.557890 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.559832 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.573651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1354/2715] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:25 2022] PUT /identity/v3/projects/13cbc80d2355452997c6d5e57b8bcc6b/users/dfd7c44ad6734801a470370321b1322c/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:25.576986 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a234b18-f73c-4bb1-b5eb-420fd1add1cf admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.584432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.584878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.585047 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.615400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] MFA Rules not processed for user `dfd7c44ad6734801a470370321b1322c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:25.616620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.638839 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.639709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.639997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.640279 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] PATH_INFO: `/v3/projects/72912e61abce48b6a0ee27f5aac00732` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.641497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=72912e61abce48b6a0ee27f5aac00732)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.643423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.651382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:25.685028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1355/2716] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:25.691542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72912e61abce48b6a0ee27f5aac00732', 'request_id': 'req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.697603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72912e61abce48b6a0ee27f5aac00732', 'request_id': 'req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.697786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.699097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.705181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72912e61abce48b6a0ee27f5aac00732', 'request_id': 'req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.717173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] The token cache is being invalidate because project 72912e61abce48b6a0ee27f5aac00732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:25.717396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.718073 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-18a7e55c-0da6-4d17-a5f9-ed34380df7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:25.719632 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1362/2717] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:25 2022] DELETE /identity/v3/projects/72912e61abce48b6a0ee27f5aac00732 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:25.724387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a7ea-33d5-41ce-a0be-2463f31dc077 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.724678 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a7ea-33d5-41ce-a0be-2463f31dc077 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.724988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5a7ea-33d5-41ce-a0be-2463f31dc077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.742398 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-03a5a7ea-33d5-41ce-a0be-2463f31dc077 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:25.743610 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-03a5a7ea-33d5-41ce-a0be-2463f31dc077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.760009 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59c719-3712-4dd0-8001-e236ccabf4cb None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.772821 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:25.804396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1363/2718] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:25.811964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b37a91-b337-436e-ac75-7b5bb41190ac None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.812259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b37a91-b337-436e-ac75-7b5bb41190ac None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.812545 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b37a91-b337-436e-ac75-7b5bb41190ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.814158 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0239f0ca3ab3423491d3844fd79a2868', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.814815 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.815100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.815379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.817306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.831746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-68b37a91-b337-436e-ac75-7b5bb41190ac None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:25.833559 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-68b37a91-b337-436e-ac75-7b5bb41190ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:25.841531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1364/2719] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:10:25.845579 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b37a91-b337-436e-ac75-7b5bb41190ac None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.854409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b37a91-b337-436e-ac75-7b5bb41190ac None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.859276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.859918 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.860204 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.860482 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.861290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:25.861607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.863208 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.878583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1365/2720] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:10:25.881963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.882693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.883741 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.892710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c5946-abac-49d3-a973-a421b9e7caec admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:25.894271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1356/2721] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:25.897546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:25.898317 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:25.898611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:25.898892 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:25.899609 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:25.901643 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:25.940392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1366/2722] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:25.944499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:25.955546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c491cd8-02c6-4749-89b2-cbf2106ec952 service glance] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.024231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.025139 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.025584 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.026131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.027064 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.029920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.053084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1357/2723] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:25 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:26.057467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.069009 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe8bce-bcb5-4581-b609-99573ec6fa62 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.074549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.075217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.075508 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.075788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.076518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.078155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.088776 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1367/2724] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:26.093078 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.105554 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390cf28-8992-49e6-89bd-8b695ca382ba admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.110803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aa745-914e-48d2-859c-c63d2e38d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.111557 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa745-914e-48d2-859c-c63d2e38d8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.111850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa745-914e-48d2-859c-c63d2e38d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.112129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa745-914e-48d2-859c-c63d2e38d8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.113022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aa745-914e-48d2-859c-c63d2e38d8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.115183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aa745-914e-48d2-859c-c63d2e38d8a9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.126125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1358/2725] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:26.130721 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.142608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc52a3-4585-4791-a367-300275431b5f admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.147928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.149109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.149405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.149685 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] PATH_INFO: `/v3/projects/c554c6123a364330afb55c514467d51d/users/ec10ba24348645f6af0a1f26f833911c/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.168500 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c554c6123a364330afb55c514467d51d, user_id=ec10ba24348645f6af0a1f26f833911c, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.171392 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.188981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1368/2726] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:26 2022] PUT /identity/v3/projects/c554c6123a364330afb55c514467d51d/users/ec10ba24348645f6af0a1f26f833911c/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:26.199868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cac0-b507-4981-b2d1-f1b81fd45ec7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.200191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cac0-b507-4981-b2d1-f1b81fd45ec7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.200438 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cac0-b507-4981-b2d1-f1b81fd45ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.230556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a942cac0-b507-4981-b2d1-f1b81fd45ec7 None None] MFA Rules not processed for user `ec10ba24348645f6af0a1f26f833911c`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:26.231625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a942cac0-b507-4981-b2d1-f1b81fd45ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:26.283266 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:26.320203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1359/2727] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:26.321054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.374501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:26.433857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89452f-c8ee-4a40-9bc1-d119062cc859 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.438729 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.439395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.439698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.439958 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.440647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.443192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.469763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1369/2728] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:26 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:26.473870 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942cac0-b507-4981-b2d1-f1b81fd45ec7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.487132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942cac0-b507-4981-b2d1-f1b81fd45ec7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.546928 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.547683 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.547900 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.548179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.548898 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.552910 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.571745 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1360/2729] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:26 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:26.575731 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.585921 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a3467-6809-4889-aace-74d738d8ca21 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.590535 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.591190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.591473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.591752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.592477 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.594110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.604438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1370/2730] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:26.608274 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.619332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3f6e6-d740-485f-882a-458f1487b210 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.624072 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6828824-2097-454d-8039-a10149882840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.625043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6828824-2097-454d-8039-a10149882840 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.625326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6828824-2097-454d-8039-a10149882840 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.625600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6828824-2097-454d-8039-a10149882840 admin admin] PATH_INFO: `/v3/projects/c8c6038d1ef7440bad4425d489ee8709/users/026cce97f2be47fb9ab364cc994e4427/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.640904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6828824-2097-454d-8039-a10149882840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c6038d1ef7440bad4425d489ee8709, user_id=026cce97f2be47fb9ab364cc994e4427, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.643559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6828824-2097-454d-8039-a10149882840 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.655799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1361/2731] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:26 2022] PUT /identity/v3/projects/c8c6038d1ef7440bad4425d489ee8709/users/026cce97f2be47fb9ab364cc994e4427/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:26.660001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.670950 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f16417-4830-41be-bf8f-480fc46f3820 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.726724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084e5c0-2032-45bf-8985-1dd00df3984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.727395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084e5c0-2032-45bf-8985-1dd00df3984e admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.727689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084e5c0-2032-45bf-8985-1dd00df3984e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.727970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084e5c0-2032-45bf-8985-1dd00df3984e admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.728692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084e5c0-2032-45bf-8985-1dd00df3984e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.730321 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084e5c0-2032-45bf-8985-1dd00df3984e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.740467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1371/2732] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:26.744740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6828824-2097-454d-8039-a10149882840 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.755053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6828824-2097-454d-8039-a10149882840 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.759693 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.760574 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.760887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.761173 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] PATH_INFO: `/v3/projects/c8c6038d1ef7440bad4425d489ee8709/users/026cce97f2be47fb9ab364cc994e4427/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.763134 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c6038d1ef7440bad4425d489ee8709, user_id=026cce97f2be47fb9ab364cc994e4427, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.765868 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.776061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1362/2733] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:26 2022] PUT /identity/v3/projects/c8c6038d1ef7440bad4425d489ee8709/users/026cce97f2be47fb9ab364cc994e4427/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:26.787812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d700d6-ec85-4abc-b9a6-42692f0d18ff None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.788133 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d700d6-ec85-4abc-b9a6-42692f0d18ff None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.788418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d700d6-ec85-4abc-b9a6-42692f0d18ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.817237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-e7d700d6-ec85-4abc-b9a6-42692f0d18ff None None] MFA Rules not processed for user `026cce97f2be47fb9ab364cc994e4427`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:26.818456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-e7d700d6-ec85-4abc-b9a6-42692f0d18ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:26.888741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1372/2734] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:26.899400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:26.901177 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:26.959968 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630b9e9-c0fa-4e74-b60e-195d0cb0d560 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:26.963889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da08bd-fde5-4aa6-9452-8fe1a2285c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:26.964586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da08bd-fde5-4aa6-9452-8fe1a2285c0e service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:26.964897 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da08bd-fde5-4aa6-9452-8fe1a2285c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:26.965199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da08bd-fde5-4aa6-9452-8fe1a2285c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:26.975634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da08bd-fde5-4aa6-9452-8fe1a2285c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:26.977123 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da08bd-fde5-4aa6-9452-8fe1a2285c0e service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:26.987029 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1363/2735] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:28.351251 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d700d6-ec85-4abc-b9a6-42692f0d18ff None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:28.369612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:28.415327 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d700d6-ec85-4abc-b9a6-42692f0d18ff None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:28.419965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:28.420742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:28.421059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:28.421338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] PATH_INFO: `/v3/users/ec10ba24348645f6af0a1f26f833911c` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:28.422599 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec10ba24348645f6af0a1f26f833911c)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:28.424531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:28.470418 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec10ba24348645f6af0a1f26f833911c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:28.471220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec10ba24348645f6af0a1f26f833911c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:28.471852 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec10ba24348645f6af0a1f26f833911c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:28.479346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec10ba24348645f6af0a1f26f833911c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:28.493138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec10ba24348645f6af0a1f26f833911c'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:28.506329 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1373/2736] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:28 2022] DELETE /identity/v3/users/ec10ba24348645f6af0a1f26f833911c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:28.510774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:28.511255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:28.511392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:28.529476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:28.530599 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:28.560796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:28.599148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1364/2737] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:28.608669 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:28.610252 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:28.668584 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c76740-26d1-4171-82e5-d8600ba11862 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:28.672217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:28.672903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:28.673192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:28.673466 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:28.682369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:28.683800 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:28.693429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1374/2738] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:28.980982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:28.982426 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:28.998419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185b53-53fe-4ab2-bbd6-35cd0ef52263 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:29.004548 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:29.006375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.006795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.007069 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] PATH_INFO: `/v3/projects/c554c6123a364330afb55c514467d51d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.008570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c554c6123a364330afb55c514467d51d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:29.010712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:29.061865 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c554c6123a364330afb55c514467d51d', 'request_id': 'req-090af29e-934f-4172-b867-8ec6406fe4c0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.067742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c554c6123a364330afb55c514467d51d', 'request_id': 'req-090af29e-934f-4172-b867-8ec6406fe4c0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.075097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c554c6123a364330afb55c514467d51d', 'request_id': 'req-090af29e-934f-4172-b867-8ec6406fe4c0'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.087163 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] The token cache is being invalidate because project c554c6123a364330afb55c514467d51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:29.087468 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.088089 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.092692 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1365/2739] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:28 2022] DELETE /identity/v3/projects/c554c6123a364330afb55c514467d51d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:29.093858 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:29.103737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23db9-c04d-41fd-838c-87b8a8fd164a service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:29.140095 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:29.165354 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:29.166245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.166538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.166817 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] PATH_INFO: `/v3/users/026cce97f2be47fb9ab364cc994e4427` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.168030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=026cce97f2be47fb9ab364cc994e4427)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:29.170008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:29.217692 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026cce97f2be47fb9ab364cc994e4427'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.218342 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026cce97f2be47fb9ab364cc994e4427'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.219004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026cce97f2be47fb9ab364cc994e4427'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.226971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026cce97f2be47fb9ab364cc994e4427'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.239940 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-e55e32ff-a867-4bec-9553-1acf8588b9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026cce97f2be47fb9ab364cc994e4427'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.253696 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1375/2740] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:29 2022] DELETE /identity/v3/users/026cce97f2be47fb9ab364cc994e4427 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:29.540204 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:29.541798 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:29.560208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090af29e-934f-4172-b867-8ec6406fe4c0 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:29.619405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:29.620201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.620489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.620770 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] PATH_INFO: `/v3/projects/c8c6038d1ef7440bad4425d489ee8709` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.621984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c6038d1ef7440bad4425d489ee8709)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:29.623881 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:29.670373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c6038d1ef7440bad4425d489ee8709', 'request_id': 'req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.675888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c6038d1ef7440bad4425d489ee8709', 'request_id': 'req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.679008 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:29.684718 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c6038d1ef7440bad4425d489ee8709', 'request_id': 'req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.698341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] The token cache is being invalidate because project c8c6038d1ef7440bad4425d489ee8709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:29.698631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.699288 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-5c3ea159-f6ff-43e2-9bf5-50d20b9a2826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:29.700814 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1366/2741] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:29 2022] DELETE /identity/v3/projects/c8c6038d1ef7440bad4425d489ee8709 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:29.705658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad775d0-9694-4249-a72a-df80fe053cba None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.706116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad775d0-9694-4249-a72a-df80fe053cba None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.706407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad775d0-9694-4249-a72a-df80fe053cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.725525 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fad775d0-9694-4249-a72a-df80fe053cba None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:29.726730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fad775d0-9694-4249-a72a-df80fe053cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:29.786781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1376/2742] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:29.794416 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7078c-1ae8-44c2-950f-9fb999bd1a5d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.794707 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7078c-1ae8-44c2-950f-9fb999bd1a5d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.794985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7078c-1ae8-44c2-950f-9fb999bd1a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.813085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-3dc7078c-1ae8-44c2-950f-9fb999bd1a5d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:29.814300 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3dc7078c-1ae8-44c2-950f-9fb999bd1a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:29.823051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1367/2743] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:29.827242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad775d0-9694-4249-a72a-df80fe053cba None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:29.837565 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad775d0-9694-4249-a72a-df80fe053cba None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:29.842106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:29.842756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.843041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.843320 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.844044 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:29.844362 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:29.846022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:29.857963 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1377/2744] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:29.862356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7078c-1ae8-44c2-950f-9fb999bd1a5d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:29.871602 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7078c-1ae8-44c2-950f-9fb999bd1a5d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:29.876143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:29.876781 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.877093 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.877372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.878057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:29.880516 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:29.906556 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1368/2745] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:29 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:29.911048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:29.921111 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bf601-6a2f-47a0-b412-bbc7617908f8 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:29.963094 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:29.979903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:29.980657 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:29.980981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:29.981269 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:29.981970 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:29.983938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.003198 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1378/2746] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:29 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:30.007854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.018363 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51837cd5-605b-4fef-8293-1bac4c1ea166 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.023751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.024422 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.024777 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.025022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.025838 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.027490 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.038190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1369/2747] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:30.042083 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.052726 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5c3f3-bc35-4301-8b25-db1bb8d9399a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.057352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f42aa-35bd-4f85-a88b-3ef18a2ea45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.058003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f42aa-35bd-4f85-a88b-3ef18a2ea45a admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.058287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f42aa-35bd-4f85-a88b-3ef18a2ea45a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.058566 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f42aa-35bd-4f85-a88b-3ef18a2ea45a admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.059289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f42aa-35bd-4f85-a88b-3ef18a2ea45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.060916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f42aa-35bd-4f85-a88b-3ef18a2ea45a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.071875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1379/2748] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:30.075886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.086127 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840a040-c0b3-4fc1-91d8-ba62c8e7a910 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.090800 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.091676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.091969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.092246 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] PATH_INFO: `/v3/projects/102eaafe3b25488499f84c1a1cf0096b/users/d56f127dc4354d09894f1f1dcdfd7187/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.106713 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102eaafe3b25488499f84c1a1cf0096b, user_id=d56f127dc4354d09894f1f1dcdfd7187, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.109415 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.122560 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1370/2749] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:30 2022] PUT /identity/v3/projects/102eaafe3b25488499f84c1a1cf0096b/users/d56f127dc4354d09894f1f1dcdfd7187/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:30.134207 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba5e47-074c-4bd9-8e9b-19e82393eb10 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.134496 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba5e47-074c-4bd9-8e9b-19e82393eb10 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.134775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba5e47-074c-4bd9-8e9b-19e82393eb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.160319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-20ba5e47-074c-4bd9-8e9b-19e82393eb10 None None] MFA Rules not processed for user `d56f127dc4354d09894f1f1dcdfd7187`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:30.161534 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-20ba5e47-074c-4bd9-8e9b-19e82393eb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:30.227640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1380/2750] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:30.232075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.294037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e506c-9ff2-4135-9eab-a4d436490a38 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.298839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.299489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.299773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.300085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.300768 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.303190 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.329944 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1371/2751] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:30 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:30.334005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba5e47-074c-4bd9-8e9b-19e82393eb10 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.344571 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba5e47-074c-4bd9-8e9b-19e82393eb10 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.400380 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.400996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.401315 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.401562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.402264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.404177 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.424531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1381/2752] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:30 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:30.429135 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.439331 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc8ef8-0e42-448f-bad0-642a4a1758cf admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.444002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.444653 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.444966 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.445245 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.445969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.447578 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.457952 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1372/2753] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:30.462103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.472469 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3443c-67e2-4325-9188-1344b83dc763 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.477647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.478541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.478837 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.479112 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] PATH_INFO: `/v3/projects/2b55098c8e3449518e190f1210933fbc/users/9ea4a9bc47454d0d8fed7f778131590d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.494119 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b55098c8e3449518e190f1210933fbc, user_id=9ea4a9bc47454d0d8fed7f778131590d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.497079 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.510810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1382/2754] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:30 2022] PUT /identity/v3/projects/2b55098c8e3449518e190f1210933fbc/users/9ea4a9bc47454d0d8fed7f778131590d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:30.515137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.527332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc3f9e-9296-43b8-8f81-8131db4c8927 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.590563 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71257c48-0ba3-4905-8e72-28562ca5d5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.591326 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71257c48-0ba3-4905-8e72-28562ca5d5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.591689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71257c48-0ba3-4905-8e72-28562ca5d5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.592065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71257c48-0ba3-4905-8e72-28562ca5d5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.592980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71257c48-0ba3-4905-8e72-28562ca5d5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.594625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71257c48-0ba3-4905-8e72-28562ca5d5c8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.596011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:30.613392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1373/2755] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:30.614850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.625237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a0d5-f1f4-4045-af2b-7b8de49bb03a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.629859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.630744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.631029 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.631305 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] PATH_INFO: `/v3/projects/2b55098c8e3449518e190f1210933fbc/users/9ea4a9bc47454d0d8fed7f778131590d/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.633283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b55098c8e3449518e190f1210933fbc, user_id=9ea4a9bc47454d0d8fed7f778131590d, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.635956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.645961 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1383/2756] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:30 2022] PUT /identity/v3/projects/2b55098c8e3449518e190f1210933fbc/users/9ea4a9bc47454d0d8fed7f778131590d/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:30.655802 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd74c6-8b5e-4309-92c3-83de6ed4151d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.656131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd74c6-8b5e-4309-92c3-83de6ed4151d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.656627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd74c6-8b5e-4309-92c3-83de6ed4151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.685148 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-b9cd74c6-8b5e-4309-92c3-83de6ed4151d None None] MFA Rules not processed for user `9ea4a9bc47454d0d8fed7f778131590d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:30.686364 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-b9cd74c6-8b5e-4309-92c3-83de6ed4151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:30.756759 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1374/2757] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:30.767544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:30.769375 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:30.776427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:30.830264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048fc8c-7a38-4b04-9cbf-8f828aa555e2 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:30.834639 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9f2e-d10d-4594-898f-d710314b0376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:30.835303 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9f2e-d10d-4594-898f-d710314b0376 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:30.835586 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9f2e-d10d-4594-898f-d710314b0376 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:30.835867 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9f2e-d10d-4594-898f-d710314b0376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:30.845432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd9f2e-d10d-4594-898f-d710314b0376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:30.847053 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd9f2e-d10d-4594-898f-d710314b0376 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:30.857271 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1384/2758] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:30.989327 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd74c6-8b5e-4309-92c3-83de6ed4151d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:31.052359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd74c6-8b5e-4309-92c3-83de6ed4151d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:31.056985 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:31.057851 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:31.058130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:31.058425 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] PATH_INFO: `/v3/users/d56f127dc4354d09894f1f1dcdfd7187` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:31.059594 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56f127dc4354d09894f1f1dcdfd7187)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:31.061551 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:31.108579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56f127dc4354d09894f1f1dcdfd7187'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.109253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56f127dc4354d09894f1f1dcdfd7187'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.116494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56f127dc4354d09894f1f1dcdfd7187'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.117175 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56f127dc4354d09894f1f1dcdfd7187'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.124659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56f127dc4354d09894f1f1dcdfd7187'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.138054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:31.143518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1375/2759] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:30 2022] DELETE /identity/v3/users/d56f127dc4354d09894f1f1dcdfd7187 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:31.148640 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:31.148965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:31.149246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:31.166711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:31.167836 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:31.230813 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1385/2760] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:31.240938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:31.242366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:31.299904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa6a59-9565-4719-a5b7-bf52b239c208 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:31.303600 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:31.304251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:31.304536 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:31.304812 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:31.314038 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:31.315462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:31.325374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1376/2761] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:31.595774 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:31.597241 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:31.609042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8157-42de-466f-bbc8-1fc63c36b68d None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:31.614819 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:31.615763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:31.616341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:31.616635 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] PATH_INFO: `/v3/projects/102eaafe3b25488499f84c1a1cf0096b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:31.618169 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=102eaafe3b25488499f84c1a1cf0096b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:31.621225 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:31.671717 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102eaafe3b25488499f84c1a1cf0096b', 'request_id': 'req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.677716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102eaafe3b25488499f84c1a1cf0096b', 'request_id': 'req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.685233 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102eaafe3b25488499f84c1a1cf0096b', 'request_id': 'req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.696711 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] The token cache is being invalidate because project 102eaafe3b25488499f84c1a1cf0096b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:31.697084 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.697716 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.699237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1386/2762] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:31 2022] DELETE /identity/v3/projects/102eaafe3b25488499f84c1a1cf0096b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:31.703179 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:31.712620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038181-c277-4369-8f8f-e27f3283b2ed service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:31.766656 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:31.767520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:31.767797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:31.768068 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] PATH_INFO: `/v3/users/9ea4a9bc47454d0d8fed7f778131590d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:31.769252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea4a9bc47454d0d8fed7f778131590d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:31.771203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:31.818762 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea4a9bc47454d0d8fed7f778131590d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.820275 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea4a9bc47454d0d8fed7f778131590d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.827739 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea4a9bc47454d0d8fed7f778131590d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.828445 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea4a9bc47454d0d8fed7f778131590d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.836192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d8210657-7495-4e40-ae7b-084e400ee113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea4a9bc47454d0d8fed7f778131590d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:31.854351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1377/2763] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:31 2022] DELETE /identity/v3/users/9ea4a9bc47454d0d8fed7f778131590d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:32.131848 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.133837 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:32.148067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453a8d7-f4d4-4cd6-95e9-e7ae75cde52e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.179226 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:32.210976 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.211786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.212068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.212340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] PATH_INFO: `/v3/projects/2b55098c8e3449518e190f1210933fbc` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.213591 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b55098c8e3449518e190f1210933fbc)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.215706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.260948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b55098c8e3449518e190f1210933fbc', 'request_id': 'req-b3e41986-f944-49b2-893d-947525b48354'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:32.266855 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b55098c8e3449518e190f1210933fbc', 'request_id': 'req-b3e41986-f944-49b2-893d-947525b48354'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:32.274679 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b55098c8e3449518e190f1210933fbc', 'request_id': 'req-b3e41986-f944-49b2-893d-947525b48354'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:32.286509 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] The token cache is being invalidate because project 2b55098c8e3449518e190f1210933fbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:32.286804 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:32.287503 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-b3e41986-f944-49b2-893d-947525b48354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:32.289078 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1387/2764] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:32 2022] DELETE /identity/v3/projects/2b55098c8e3449518e190f1210933fbc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:32.294121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788834c-9561-4550-97c3-7935f7322024 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.294441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788834c-9561-4550-97c3-7935f7322024 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.294740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788834c-9561-4550-97c3-7935f7322024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.314164 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-c788834c-9561-4550-97c3-7935f7322024 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:32.315400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c788834c-9561-4550-97c3-7935f7322024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:32.333016 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:32.377214 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1378/2765] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:32.386481 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22677bed-9026-4226-95b8-0eca61e47d39 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.386835 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22677bed-9026-4226-95b8-0eca61e47d39 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.387127 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22677bed-9026-4226-95b8-0eca61e47d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.405264 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-22677bed-9026-4226-95b8-0eca61e47d39 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:32.406613 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-22677bed-9026-4226-95b8-0eca61e47d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:32.414847 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1388/2766] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:32.419219 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788834c-9561-4550-97c3-7935f7322024 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.430690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788834c-9561-4550-97c3-7935f7322024 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.435409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.436075 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.436358 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.436633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.437398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:32.437710 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.439302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.452400 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1379/2767] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:10:32.456801 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22677bed-9026-4226-95b8-0eca61e47d39 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.466498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22677bed-9026-4226-95b8-0eca61e47d39 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.471430 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.472099 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.472388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.472668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.473405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.475343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.501386 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1389/2768] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:32 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:32.506396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.517704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf01e8-fdab-4957-8216-cd712bd96506 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.577117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.577796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.578110 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.578392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.579156 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.581112 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.602046 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1380/2769] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:32 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:32.609281 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.622844 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eff653-201e-4279-8e80-5b4bc35acabb admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.627558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.628225 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.628615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.628988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.630352 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.632101 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.643638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1390/2770] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:32.647715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.659212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7c376-b6e2-4495-b6bf-d5883ae962f9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.664419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af784cf5-ab2d-47e4-883a-cfcba64b7887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.665097 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af784cf5-ab2d-47e4-883a-cfcba64b7887 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.665380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af784cf5-ab2d-47e4-883a-cfcba64b7887 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.665652 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af784cf5-ab2d-47e4-883a-cfcba64b7887 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.666377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af784cf5-ab2d-47e4-883a-cfcba64b7887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.667961 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af784cf5-ab2d-47e4-883a-cfcba64b7887 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.684890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1381/2771] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:32.687108 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.697650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f768c33-1056-47e1-b888-f179e8ffd166 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.703407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.704290 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.704954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.704954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] PATH_INFO: `/v3/projects/c746dce1588b496988840544e27bb448/users/490846187bf9498d83ada5e8001e9a0d/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.720499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c746dce1588b496988840544e27bb448, user_id=490846187bf9498d83ada5e8001e9a0d, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.723272 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.737555 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1391/2772] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:32 2022] PUT /identity/v3/projects/c746dce1588b496988840544e27bb448/users/490846187bf9498d83ada5e8001e9a0d/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:32.749122 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a7b57-ce5b-4a4b-85ff-e7eaa372fc42 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.749616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a7b57-ce5b-4a4b-85ff-e7eaa372fc42 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.749823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a7b57-ce5b-4a4b-85ff-e7eaa372fc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.777251 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-fc4a7b57-ce5b-4a4b-85ff-e7eaa372fc42 None None] MFA Rules not processed for user `490846187bf9498d83ada5e8001e9a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:32.778476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fc4a7b57-ce5b-4a4b-85ff-e7eaa372fc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:32.841784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1382/2773] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:32.846339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.904979 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:32.908965 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738e8c3-76a0-4635-ae1b-3f3fc11f3cc0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.915063 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:32.915877 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:32.916378 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:32.916706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:32.917564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:32.919484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:32.944287 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1392/2774] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:32 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:32.948455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a7b57-ce5b-4a4b-85ff-e7eaa372fc42 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:32.959085 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a7b57-ce5b-4a4b-85ff-e7eaa372fc42 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:32.970950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:33.015970 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.016643 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.016956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.017257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.017939 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.019834 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.040125 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1383/2775] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:32 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:33.044138 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.055408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa7f2e-dcc6-4f9b-846e-9563a8b10987 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.060052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.060720 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.061037 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.061318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.062066 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.063653 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.074212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1393/2776] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:33.078227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.088757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b2686-873a-4af6-b36c-a194efb8df84 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.093392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdddaf-cc92-47e8-bd6a-3e2a43c67c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.094051 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdddaf-cc92-47e8-bd6a-3e2a43c67c63 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.094332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdddaf-cc92-47e8-bd6a-3e2a43c67c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.094628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdddaf-cc92-47e8-bd6a-3e2a43c67c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.095377 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdddaf-cc92-47e8-bd6a-3e2a43c67c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.096976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdddaf-cc92-47e8-bd6a-3e2a43c67c63 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.107545 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1384/2777] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:33.111519 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.121402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946ce08-fa58-4caa-83c7-4633a336ba2c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.125917 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.126786 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.127067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.127391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] PATH_INFO: `/v3/projects/f164a6afa85c440d90b32b09efa9052d/users/d8118f94105a4928ae56c9632e393983/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.144687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f164a6afa85c440d90b32b09efa9052d, user_id=d8118f94105a4928ae56c9632e393983, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.148052 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.162194 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1394/2778] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:33 2022] PUT /identity/v3/projects/f164a6afa85c440d90b32b09efa9052d/users/d8118f94105a4928ae56c9632e393983/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:33.172372 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed9bfe-b8e7-44ac-84d7-4f2a9bbc6747 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.172659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed9bfe-b8e7-44ac-84d7-4f2a9bbc6747 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.172969 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed9bfe-b8e7-44ac-84d7-4f2a9bbc6747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.200043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-beed9bfe-b8e7-44ac-84d7-4f2a9bbc6747 None None] MFA Rules not processed for user `d8118f94105a4928ae56c9632e393983`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:33.201323 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-beed9bfe-b8e7-44ac-84d7-4f2a9bbc6747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:33.269002 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1385/2779] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:33.273996 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.337075 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffa8a2-57d0-4aa1-9326-57432c179f48 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.341931 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.342604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.342888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.343167 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.343869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.345854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.370826 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1395/2780] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:33 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:33.374934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed9bfe-b8e7-44ac-84d7-4f2a9bbc6747 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.386719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed9bfe-b8e7-44ac-84d7-4f2a9bbc6747 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.442918 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.443591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.443876 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.444183 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.444926 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.446826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.466037 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1386/2781] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:33 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:33.470041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.480740 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08795d19-f0e7-4419-9007-f0977b74d051 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.485401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.485997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.486283 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.486561 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.487285 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.488928 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.500038 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1396/2782] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:10:33.503978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.513801 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d59ee-5bf0-4e38-a926-f0cb307e7c6f admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.518335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.519201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.519482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.519755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] PATH_INFO: `/v3/projects/ad25cc1a4f4b4f2499047acef6c554c2/users/1c1c275c4a1e401ebc355d7874cc84dc/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.531504 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:33.534935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad25cc1a4f4b4f2499047acef6c554c2, user_id=1c1c275c4a1e401ebc355d7874cc84dc, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.538261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.552441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1387/2783] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:33 2022] PUT /identity/v3/projects/ad25cc1a4f4b4f2499047acef6c554c2/users/1c1c275c4a1e401ebc355d7874cc84dc/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:33.556782 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.566834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa7f61-c209-4a0d-bf3a-92bbfda81440 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.622580 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9713d0d-6abc-4cba-b292-67b985e5e306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.623540 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9713d0d-6abc-4cba-b292-67b985e5e306 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.623834 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9713d0d-6abc-4cba-b292-67b985e5e306 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.624118 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9713d0d-6abc-4cba-b292-67b985e5e306 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.624898 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9713d0d-6abc-4cba-b292-67b985e5e306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.626506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9713d0d-6abc-4cba-b292-67b985e5e306 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.635980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1397/2784] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:33.640080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.650263 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bf8b6-9aed-4498-b69f-a30896aeca01 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.655092 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.655973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.656256 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.656534 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] PATH_INFO: `/v3/projects/ad25cc1a4f4b4f2499047acef6c554c2/users/1c1c275c4a1e401ebc355d7874cc84dc/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.658830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad25cc1a4f4b4f2499047acef6c554c2, user_id=1c1c275c4a1e401ebc355d7874cc84dc, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.662057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.672369 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1388/2785] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:33 2022] PUT /identity/v3/projects/ad25cc1a4f4b4f2499047acef6c554c2/users/1c1c275c4a1e401ebc355d7874cc84dc/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:33.682810 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.683110 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.683396 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.709247 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] MFA Rules not processed for user `1c1c275c4a1e401ebc355d7874cc84dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:33.710552 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:33.716822 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:33.786049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1398/2786] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:33.799663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.802133 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:33.860806 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142af5ed-02e0-454c-8a18-f9606908975e admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.864938 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.865686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.865970 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.866241 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:33.875829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:33.877339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:33.887937 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1389/2787] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:33.927237 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:33.928882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:33.941439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c145f-09ba-4398-9b79-75198195dae8 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:33.945617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0c930-5a3c-4ca4-b12a-cbb425529dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:33.946262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c930-5a3c-4ca4-b12a-cbb425529dfa service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:33.946544 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c930-5a3c-4ca4-b12a-cbb425529dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:33.946829 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c930-5a3c-4ca4-b12a-cbb425529dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:34.005532 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0c930-5a3c-4ca4-b12a-cbb425529dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:34.006884 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0c930-5a3c-4ca4-b12a-cbb425529dfa service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:34.015794 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1399/2788] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:37.984734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:38.056638 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3faf87-b105-4065-8c08-fbd51b3ea509 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:38.061215 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:38.062121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:38.062401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:38.062676 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] PATH_INFO: `/v3/users/490846187bf9498d83ada5e8001e9a0d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:38.063861 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=490846187bf9498d83ada5e8001e9a0d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:38.065816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:38.110724 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '490846187bf9498d83ada5e8001e9a0d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.111301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '490846187bf9498d83ada5e8001e9a0d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.118227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '490846187bf9498d83ada5e8001e9a0d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.118856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '490846187bf9498d83ada5e8001e9a0d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.125671 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '490846187bf9498d83ada5e8001e9a0d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.139198 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:38.144712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1390/2789] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:37 2022] DELETE /identity/v3/users/490846187bf9498d83ada5e8001e9a0d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:38.149628 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:38.150423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:38.150423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:38.167742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:38.168971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:38.228623 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1400/2790] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:38.238419 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:38.239859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:38.301146 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ad502-71b0-4803-bd5b-1fea2380a9b4 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:38.305398 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:38.306054 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:38.306342 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:38.306616 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:38.315915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:38.317388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:38.327793 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1391/2791] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:38.607608 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:38.609259 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:38.624491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500973d1-7f91-4c90-9a56-ef225acc9966 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:38.629458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:38.630343 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:38.630629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:38.630903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] PATH_INFO: `/v3/projects/c746dce1588b496988840544e27bb448` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:38.632087 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c746dce1588b496988840544e27bb448)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:38.634040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:38.676012 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:38.687196 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c746dce1588b496988840544e27bb448', 'request_id': 'req-42b48404-20d3-444d-8c3c-e773b106f79a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.693341 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c746dce1588b496988840544e27bb448', 'request_id': 'req-42b48404-20d3-444d-8c3c-e773b106f79a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.700753 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c746dce1588b496988840544e27bb448', 'request_id': 'req-42b48404-20d3-444d-8c3c-e773b106f79a'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.712123 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] The token cache is being invalidate because project c746dce1588b496988840544e27bb448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:38.712416 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.713072 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.714611 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1401/2792] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:38 2022] DELETE /identity/v3/projects/c746dce1588b496988840544e27bb448 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:38.718595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:38.729351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00687bb-c36c-44d2-829e-3faee090f1d5 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:38.785674 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:38.786546 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:38.786826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:38.787096 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] PATH_INFO: `/v3/users/d8118f94105a4928ae56c9632e393983` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:38.788847 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8118f94105a4928ae56c9632e393983)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:38.790799 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:38.841076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8118f94105a4928ae56c9632e393983'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.841076 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8118f94105a4928ae56c9632e393983'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.847825 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8118f94105a4928ae56c9632e393983'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.848477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8118f94105a4928ae56c9632e393983'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.856409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8118f94105a4928ae56c9632e393983'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:38.874784 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1392/2793] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:38 2022] DELETE /identity/v3/users/d8118f94105a4928ae56c9632e393983 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:39.169399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:39.170938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:39.191461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48404-20d3-444d-8c3c-e773b106f79a admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:39.260179 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:39.261223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:39.261506 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:39.261783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] PATH_INFO: `/v3/projects/f164a6afa85c440d90b32b09efa9052d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:39.262982 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f164a6afa85c440d90b32b09efa9052d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:39.265032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:39.318993 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f164a6afa85c440d90b32b09efa9052d', 'request_id': 'req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.324850 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f164a6afa85c440d90b32b09efa9052d', 'request_id': 'req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.332668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f164a6afa85c440d90b32b09efa9052d', 'request_id': 'req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.346173 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] The token cache is being invalidate because project f164a6afa85c440d90b32b09efa9052d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:39.346459 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.347240 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.348724 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1402/2794] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:39 2022] DELETE /identity/v3/projects/f164a6afa85c440d90b32b09efa9052d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:39.353356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:39.354930 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:39.369264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001633-abc1-46fe-95bc-42106829e7e8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:39.390984 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:39.445830 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:39.446903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:39.447224 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:39.447828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] PATH_INFO: `/v3/users/1c1c275c4a1e401ebc355d7874cc84dc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:39.449115 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c1c275c4a1e401ebc355d7874cc84dc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:39.451170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:39.514447 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c1c275c4a1e401ebc355d7874cc84dc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.515284 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c1c275c4a1e401ebc355d7874cc84dc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.525157 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c1c275c4a1e401ebc355d7874cc84dc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.525896 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c1c275c4a1e401ebc355d7874cc84dc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.534655 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-35a0184e-8fbf-4483-971e-e5f49c5f634d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c1c275c4a1e401ebc355d7874cc84dc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.554317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1393/2795] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:10:39 2022] DELETE /identity/v3/users/1c1c275c4a1e401ebc355d7874cc84dc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:39.829857 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:39.831403 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:39.848859 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30ab92-9453-4b71-a26a-a1e12b64f5ba admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:39.918147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:39.925427 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:39.926476 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:39.926944 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:39.927564 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] PATH_INFO: `/v3/projects/ad25cc1a4f4b4f2499047acef6c554c2` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:39.929408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad25cc1a4f4b4f2499047acef6c554c2)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:39.931651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:39.981258 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad25cc1a4f4b4f2499047acef6c554c2', 'request_id': 'req-3db6dc32-f074-437c-8128-9b5c79e0e3ab'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.987223 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad25cc1a4f4b4f2499047acef6c554c2', 'request_id': 'req-3db6dc32-f074-437c-8128-9b5c79e0e3ab'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:39.995483 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad25cc1a4f4b4f2499047acef6c554c2', 'request_id': 'req-3db6dc32-f074-437c-8128-9b5c79e0e3ab'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:40.006645 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] The token cache is being invalidate because project ad25cc1a4f4b4f2499047acef6c554c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:10:40.006935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:40.007604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-3db6dc32-f074-437c-8128-9b5c79e0e3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:10:40.013771 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1403/2796] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:10:39 2022] DELETE /identity/v3/projects/ad25cc1a4f4b4f2499047acef6c554c2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:40.015579 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e6ba-83d4-4922-87c7-d9c48d31efb5 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.015779 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e6ba-83d4-4922-87c7-d9c48d31efb5 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.016083 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e6ba-83d4-4922-87c7-d9c48d31efb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.035795 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-a931e6ba-83d4-4922-87c7-d9c48d31efb5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:40.037050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-a931e6ba-83d4-4922-87c7-d9c48d31efb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:40.099482 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1394/2797] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:40.108429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010b7f0-0353-459c-a539-97e32239f559 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.108722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010b7f0-0353-459c-a539-97e32239f559 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.109042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010b7f0-0353-459c-a539-97e32239f559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.128239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-b010b7f0-0353-459c-a539-97e32239f559 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:40.129652 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-b010b7f0-0353-459c-a539-97e32239f559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:40.137706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1404/2798] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:40.141826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e6ba-83d4-4922-87c7-d9c48d31efb5 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:40.151888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e6ba-83d4-4922-87c7-d9c48d31efb5 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:40.156401 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:40.157111 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.157443 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.157751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.158785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:10:40.159137 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:40.160850 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:40.178423 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1395/2799] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:10:40.183183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010b7f0-0353-459c-a539-97e32239f559 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:40.192594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010b7f0-0353-459c-a539-97e32239f559 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:40.197444 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:40.198114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.198397 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.198681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.199400 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:40.201422 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:40.227907 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1405/2800] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:40 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:40.232011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:40.241823 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b48db-9742-4f5d-8748-0b5c42a1ad08 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:40.275414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:40.306917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:40.307957 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.308242 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.308512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.309239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:40.311147 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:40.330339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1396/2801] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:40 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:40.335074 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:40.346328 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad27b7-b317-40ec-a3e2-6f890a2effa6 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:40.351515 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:40.352455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.352741 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.353041 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.353762 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:40.355350 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:40.365549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1406/2802] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:40.369697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:40.380053 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a82b44-a1a4-4a17-9d1b-104555950d96 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:40.384751 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90a648-0b04-4825-aabc-2e2f0143b62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:40.385512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90a648-0b04-4825-aabc-2e2f0143b62f admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.385845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90a648-0b04-4825-aabc-2e2f0143b62f admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.386128 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90a648-0b04-4825-aabc-2e2f0143b62f admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.386888 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90a648-0b04-4825-aabc-2e2f0143b62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:40.388502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90a648-0b04-4825-aabc-2e2f0143b62f admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:40.399524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1397/2803] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:40.403737 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:40.414408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64537ce-2dae-4e3d-8fe7-d451c6d6257b admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:40.419077 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:40.419951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.420235 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.420513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] PATH_INFO: `/v3/projects/998d0cfadd2a4ec183813218c3df2260/users/0557055ffdea433595c0c8a746db3ac1/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.434957 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=998d0cfadd2a4ec183813218c3df2260, user_id=0557055ffdea433595c0c8a746db3ac1, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:40.437651 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:40.450151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1407/2804] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:40 2022] PUT /identity/v3/projects/998d0cfadd2a4ec183813218c3df2260/users/0557055ffdea433595c0c8a746db3ac1/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:10:40.455143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532af71-62ca-43fa-9ac9-bfb50537e594 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.455435 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532af71-62ca-43fa-9ac9-bfb50537e594 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.455715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532af71-62ca-43fa-9ac9-bfb50537e594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.473672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-6532af71-62ca-43fa-9ac9-bfb50537e594 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:40.474991 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-6532af71-62ca-43fa-9ac9-bfb50537e594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:40.534212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1398/2805] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:40.544297 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:40.546116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:40.609681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159aa41-953e-4e70-a3bc-4b4614d42ef0 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:40.613553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:40.614211 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:40.614498 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:40.614775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:40.625882 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:40.627360 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:40.639325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1408/2806] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:41.251074 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ba8d-6429-43d2-a6de-1a637322c355 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:41.251365 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ba8d-6429-43d2-a6de-1a637322c355 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:41.251679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730ba8d-6429-43d2-a6de-1a637322c355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:41.278514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-2730ba8d-6429-43d2-a6de-1a637322c355 None None] MFA Rules not processed for user `0557055ffdea433595c0c8a746db3ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:41.279615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2730ba8d-6429-43d2-a6de-1a637322c355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:41.347612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1399/2807] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:41.353021 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:41.371222 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8081092-54d1-4fc1-8080-89a547ee1fad service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:41.375748 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:41.376401 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:41.376683 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:41.376999 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:41.377730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:41.379624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:41.409823 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1409/2808] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:10:41 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:10:41.411325 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730ba8d-6429-43d2-a6de-1a637322c355 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:41.424518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730ba8d-6429-43d2-a6de-1a637322c355 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:41.458672 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:41.489983 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:41.491213 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:41.491499 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:41.491828 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:41.492526 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:41.494558 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:41.515531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1400/2809] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:10:41 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:10:41.519538 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:41.525547 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:41.532601 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd12445-3a23-4a0f-8794-70099cc1b736 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:41.539263 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:41.539954 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:41.540246 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:41.540526 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:41.541298 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:41.542903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:41.555209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1410/2810] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:41.559911 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:41.570509 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1bb94-b665-42b9-bae4-856826a12ac6 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:41.575387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:41.576280 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:41.576594 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:41.576906 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] PATH_INFO: `/v3/projects/9d14d7e681b84e3da4d8b93c76bcb654/users/ea75d8a844114c84ac75ca06a582daa9/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:41.592011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d14d7e681b84e3da4d8b93c76bcb654, user_id=ea75d8a844114c84ac75ca06a582daa9, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:41.594747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:41.609100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1401/2811] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:41 2022] PUT /identity/v3/projects/9d14d7e681b84e3da4d8b93c76bcb654/users/ea75d8a844114c84ac75ca06a582daa9/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:41.612916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:41.623755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466abfd-b071-4a97-ad75-1b90d972e5ce admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:41.680676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160ed25-aae4-452e-963b-f275908b9a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:41.681359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160ed25-aae4-452e-963b-f275908b9a5b admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:41.681649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160ed25-aae4-452e-963b-f275908b9a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:41.681925 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160ed25-aae4-452e-963b-f275908b9a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:41.682647 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160ed25-aae4-452e-963b-f275908b9a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:41.684242 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160ed25-aae4-452e-963b-f275908b9a5b admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:41.694013 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1411/2812] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:10:41.698071 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:41.708161 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40099889-84e6-41e7-b04d-fb49f1464fce admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:41.713252 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:41.714126 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:41.714408 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:41.714684 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] PATH_INFO: `/v3/projects/9d14d7e681b84e3da4d8b93c76bcb654/users/ea75d8a844114c84ac75ca06a582daa9/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:41.716629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d14d7e681b84e3da4d8b93c76bcb654, user_id=ea75d8a844114c84ac75ca06a582daa9, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:41.719301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:41.730022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1402/2813] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:10:41 2022] PUT /identity/v3/projects/9d14d7e681b84e3da4d8b93c76bcb654/users/ea75d8a844114c84ac75ca06a582daa9/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:10:42.380694 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e01e1a-99dc-4519-bb16-7574ab9d9585 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:42.382218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e01e1a-99dc-4519-bb16-7574ab9d9585 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:42.382218 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e01e1a-99dc-4519-bb16-7574ab9d9585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:42.408747 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-89e01e1a-99dc-4519-bb16-7574ab9d9585 None None] MFA Rules not processed for user `ea75d8a844114c84ac75ca06a582daa9`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:10:42.410001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-89e01e1a-99dc-4519-bb16-7574ab9d9585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:42.485307 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1412/2814] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:10:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:10:42.497634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:42.499413 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:10:42.561243 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f462b-94f0-487f-8864-15c79ca74615 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:42.564809 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:42.565489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:42.565774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:42.566057 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:42.630334 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:42.632230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:42.643007 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1403/2815] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:45.054417 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e01e1a-99dc-4519-bb16-7574ab9d9585 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:10:45.063981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e01e1a-99dc-4519-bb16-7574ab9d9585 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:10:45.067553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:10:45.068195 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:10:45.069022 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:10:45.069438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:10:45.079134 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:10:45.080587 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:10:45.090919 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1413/2816] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:10:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:10:52.638842 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:10:55.086497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=73 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:06.206837 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:06.208290 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:06.304485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3d92-5028-4272-8eed-a34b75d1b0f7 service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:06.310356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:06.311020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:06.311307 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:06.311588 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:06.313410 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:06.325783 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:06.327379 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:06.338267 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1404/2817] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:06.768625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:06.835423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07994d-118c-4016-8f7d-25a00e6d8e54 service cinder] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:06.840300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:06.841220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:06.841527 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:06.841848 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] PATH_INFO: `/v3/users/0557055ffdea433595c0c8a746db3ac1` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:06.843148 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0557055ffdea433595c0c8a746db3ac1)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:06.845103 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:06.892583 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0557055ffdea433595c0c8a746db3ac1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:06.893190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0557055ffdea433595c0c8a746db3ac1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:06.893806 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0557055ffdea433595c0c8a746db3ac1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:06.900578 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0557055ffdea433595c0c8a746db3ac1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:06.912606 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0557055ffdea433595c0c8a746db3ac1'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:06.926080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1414/2818] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:06 2022] DELETE /identity/v3/users/0557055ffdea433595c0c8a746db3ac1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:07.118585 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:07.130886 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec26d9e-2f5e-485c-a0b6-059d19d1c938 service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:07.190041 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:07.190916 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:07.191203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:07.191481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] PATH_INFO: `/v3/projects/998d0cfadd2a4ec183813218c3df2260` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:07.192665 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=998d0cfadd2a4ec183813218c3df2260)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:07.194607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:07.244620 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998d0cfadd2a4ec183813218c3df2260', 'request_id': 'req-c8b39dd8-0e21-4080-9587-0e896f5aaf47'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.250205 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998d0cfadd2a4ec183813218c3df2260', 'request_id': 'req-c8b39dd8-0e21-4080-9587-0e896f5aaf47'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.260529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998d0cfadd2a4ec183813218c3df2260', 'request_id': 'req-c8b39dd8-0e21-4080-9587-0e896f5aaf47'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.268807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:07.275462 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] The token cache is being invalidate because project 998d0cfadd2a4ec183813218c3df2260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:07.276439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.276891 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.278576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1405/2819] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:07 2022] DELETE /identity/v3/projects/998d0cfadd2a4ec183813218c3df2260 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:07.282759 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:07.284677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:07.299689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e75f9-c24b-49fd-9c5f-81fbd3e600fc admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:07.358539 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:07.359431 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:07.359718 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:07.360014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] PATH_INFO: `/v3/users/ea75d8a844114c84ac75ca06a582daa9` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:07.361243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea75d8a844114c84ac75ca06a582daa9)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:07.363248 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:07.384938 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:07.410291 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea75d8a844114c84ac75ca06a582daa9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.410897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea75d8a844114c84ac75ca06a582daa9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.411518 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea75d8a844114c84ac75ca06a582daa9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.418981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea75d8a844114c84ac75ca06a582daa9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.431221 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-253eed3a-a168-42b0-adc4-b85f3a9983ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea75d8a844114c84ac75ca06a582daa9'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.444755 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1415/2820] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:07 2022] DELETE /identity/v3/users/ea75d8a844114c84ac75ca06a582daa9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:07.611826 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:07.613366 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:07.626871 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b39dd8-0e21-4080-9587-0e896f5aaf47 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:07.684586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:07.685420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:07.685704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:07.685980 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] PATH_INFO: `/v3/projects/9d14d7e681b84e3da4d8b93c76bcb654` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:07.687169 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d14d7e681b84e3da4d8b93c76bcb654)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:07.689098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:07.734004 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d14d7e681b84e3da4d8b93c76bcb654', 'request_id': 'req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.739450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d14d7e681b84e3da4d8b93c76bcb654', 'request_id': 'req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.747199 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d14d7e681b84e3da4d8b93c76bcb654', 'request_id': 'req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.758728 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] The token cache is being invalidate because project 9d14d7e681b84e3da4d8b93c76bcb654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:07.759020 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.759646 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-8067c5f9-744e-4f8d-bfcc-33a99d47f4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:07.761234 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1406/2821] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:07 2022] DELETE /identity/v3/projects/9d14d7e681b84e3da4d8b93c76bcb654 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:07.766100 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a08c1-d261-4c8e-8de3-343a115cd830 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:07.766394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a08c1-d261-4c8e-8de3-343a115cd830 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:07.766675 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a08c1-d261-4c8e-8de3-343a115cd830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:07.784725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-440a08c1-d261-4c8e-8de3-343a115cd830 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:07.785945 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-440a08c1-d261-4c8e-8de3-343a115cd830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:07.845402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1416/2822] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:07.853626 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8e5f4-6c3a-426f-831d-2457ff089bc4 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:07.853920 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8e5f4-6c3a-426f-831d-2457ff089bc4 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:07.854202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8e5f4-6c3a-426f-831d-2457ff089bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:07.873569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-24d8e5f4-6c3a-426f-831d-2457ff089bc4 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:07.874778 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-24d8e5f4-6c3a-426f-831d-2457ff089bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:07.883457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1407/2823] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:07.887725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a08c1-d261-4c8e-8de3-343a115cd830 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:07.897973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a08c1-d261-4c8e-8de3-343a115cd830 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:07.902769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:07.903471 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:07.903758 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:07.904040 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:07.904767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:11:07.905121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:07.906725 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:07.919300 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1417/2824] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:11:07.923520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8e5f4-6c3a-426f-831d-2457ff089bc4 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:07.932679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8e5f4-6c3a-426f-831d-2457ff089bc4 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:07.937538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:07.938130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:07.938412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:07.938690 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:07.939380 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:07.941374 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:07.966513 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1408/2825] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:11:07 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:07.970541 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:07.981080 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d0ff4-e99d-4d3e-acb2-018e7aa30805 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.036668 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.037372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.037693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.038468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.039165 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.041121 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.060594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1418/2826] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:11:07 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:08.064679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.076455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8384cf-6741-4977-b2bf-b73cd0085202 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.081193 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.081856 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.082140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.082417 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.083143 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.084757 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.095121 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1409/2827] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:08.098972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.109000 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933a1bb-8877-411e-92cd-9b9848a6278e admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.115373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e587bc-0c40-4b67-9c3b-a779507b2f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.115373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e587bc-0c40-4b67-9c3b-a779507b2f6e admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.115373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e587bc-0c40-4b67-9c3b-a779507b2f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.115373 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e587bc-0c40-4b67-9c3b-a779507b2f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.116312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e587bc-0c40-4b67-9c3b-a779507b2f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.117900 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e587bc-0c40-4b67-9c3b-a779507b2f6e admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.131172 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1419/2828] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:08.133028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.143566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8cfad-551d-482c-9780-cb5678fff0e5 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.148231 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.149151 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.149440 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.149736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] PATH_INFO: `/v3/projects/4b951748299c45f79056a092c4a1153d/users/ba946c8b356345f18f57156b0f047916/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.167963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b951748299c45f79056a092c4a1153d, user_id=ba946c8b356345f18f57156b0f047916, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.174129 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.189409 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1410/2829] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:11:08 2022] PUT /identity/v3/projects/4b951748299c45f79056a092c4a1153d/users/ba946c8b356345f18f57156b0f047916/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:11:08.201557 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d38de-fa65-46a5-9908-573db3fe58ad None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.202612 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d38de-fa65-46a5-9908-573db3fe58ad None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.202939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d38de-fa65-46a5-9908-573db3fe58ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.210032 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:08.232067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-444d38de-fa65-46a5-9908-573db3fe58ad None None] MFA Rules not processed for user `ba946c8b356345f18f57156b0f047916`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:08.233322 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-444d38de-fa65-46a5-9908-573db3fe58ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:08.297490 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1420/2830] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:11:08.301887 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.332021 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:08.367697 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b6a55-2c0a-484a-98c8-6d7d6dbb17fb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.372464 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.373159 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.373448 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.373726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.374429 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.376335 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.403935 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1411/2831] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:11:08 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:08.407909 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d38de-fa65-46a5-9908-573db3fe58ad None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.419478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d38de-fa65-46a5-9908-573db3fe58ad None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.483432 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.484146 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.484437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.484722 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.485470 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.487408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.510633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1421/2832] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:11:08 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:08.514612 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.526561 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988e412-70a0-4014-b8bd-a6c130109598 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.531289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.531943 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.532271 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.532559 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.533332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.535207 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.547001 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1412/2833] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:08.551262 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.561784 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912db91-86ec-41f6-9935-5f6521bf90b4 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.566968 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.567825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.568106 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.568383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] PATH_INFO: `/v3/projects/19a3992aafe448e8be21098e46ecf751/users/4bdb089e6ac241c5aa55d660ac646312/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.583312 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a3992aafe448e8be21098e46ecf751, user_id=4bdb089e6ac241c5aa55d660ac646312, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.586604 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.600546 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1422/2834] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:11:08 2022] PUT /identity/v3/projects/19a3992aafe448e8be21098e46ecf751/users/4bdb089e6ac241c5aa55d660ac646312/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:08.604934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.615432 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea56735-1162-49e0-b427-b6c30604c455 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.672208 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f078f-dea7-4e1c-bf34-a5daabbdf0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.672934 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f078f-dea7-4e1c-bf34-a5daabbdf0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.673226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f078f-dea7-4e1c-bf34-a5daabbdf0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.673507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f078f-dea7-4e1c-bf34-a5daabbdf0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.674264 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f078f-dea7-4e1c-bf34-a5daabbdf0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.675864 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f078f-dea7-4e1c-bf34-a5daabbdf0b3 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.686356 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1413/2835] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:08.690437 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.701648 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247671a-6c4a-46c2-9f1d-7466e54dcb65 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.706231 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.707109 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.707399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.707681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] PATH_INFO: `/v3/projects/19a3992aafe448e8be21098e46ecf751/users/4bdb089e6ac241c5aa55d660ac646312/roles/c21668b08994436ca32efab183c5ca03` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.709706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a3992aafe448e8be21098e46ecf751, user_id=4bdb089e6ac241c5aa55d660ac646312, role_id=c21668b08994436ca32efab183c5ca03)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:08.712377 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:08.722592 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1423/2836] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:11:08 2022] PUT /identity/v3/projects/19a3992aafe448e8be21098e46ecf751/users/4bdb089e6ac241c5aa55d660ac646312/roles/c21668b08994436ca32efab183c5ca03 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:08.784056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.784350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.784631 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:08.813268 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] MFA Rules not processed for user `4bdb089e6ac241c5aa55d660ac646312`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:08.814479 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:08.880832 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1414/2837] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:08.892014 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:08.893825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:08.948436 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:08.954828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6f42-de6f-4557-823c-be30a928cba2 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:08.958777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdde0b-72ea-4f71-95f8-4d627d83b943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:08.959456 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdde0b-72ea-4f71-95f8-4d627d83b943 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:08.959723 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdde0b-72ea-4f71-95f8-4d627d83b943 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:08.960002 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdde0b-72ea-4f71-95f8-4d627d83b943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:09.018865 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdde0b-72ea-4f71-95f8-4d627d83b943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:09.020318 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdde0b-72ea-4f71-95f8-4d627d83b943 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:09.029224 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1424/2838] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:10.383519 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:10.385282 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:10.436405 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:10.493088 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e51c7-366a-413e-be37-cc2c0384513b None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:10.497994 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:10.499225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:10.499522 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:10.499786 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] PATH_INFO: `/v3/users/dfd7c44ad6734801a470370321b1322c` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:10.501011 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd7c44ad6734801a470370321b1322c)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:10.502927 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:10.548706 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd7c44ad6734801a470370321b1322c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:10.549388 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd7c44ad6734801a470370321b1322c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:10.557420 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd7c44ad6734801a470370321b1322c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:10.558061 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd7c44ad6734801a470370321b1322c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:10.564932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd7c44ad6734801a470370321b1322c'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:10.581735 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1415/2839] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:10 2022] DELETE /identity/v3/users/dfd7c44ad6734801a470370321b1322c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:10.586266 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:10.586744 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:10.586875 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:10.604203 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:10.605461 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:10.664624 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1425/2840] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:10.674230 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:10.675629 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:10.734324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51638a-b256-451a-97d5-94f31860edb9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:10.738120 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:10.738839 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:10.739192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:10.739538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:10.749730 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:10.751257 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:10.761941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1416/2841] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:11.091803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:11.093439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:11.107363 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e094-d632-4d89-bb98-5a41355e322c None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:11.112136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:11.113122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:11.113407 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:11.113688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] PATH_INFO: `/v3/projects/13cbc80d2355452997c6d5e57b8bcc6b` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:11.114916 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=13cbc80d2355452997c6d5e57b8bcc6b)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:11.117629 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:11.171808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13cbc80d2355452997c6d5e57b8bcc6b', 'request_id': 'req-1f8375ee-7db4-433a-b9f6-8523117c2cbd'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:11.177556 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13cbc80d2355452997c6d5e57b8bcc6b', 'request_id': 'req-1f8375ee-7db4-433a-b9f6-8523117c2cbd'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:11.186250 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13cbc80d2355452997c6d5e57b8bcc6b', 'request_id': 'req-1f8375ee-7db4-433a-b9f6-8523117c2cbd'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:11.198830 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] The token cache is being invalidate because project 13cbc80d2355452997c6d5e57b8bcc6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:11.199116 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:11.199773 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:11.201388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1426/2842] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:11 2022] DELETE /identity/v3/projects/13cbc80d2355452997c6d5e57b8bcc6b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:14.302000 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:14.304084 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:14.352642 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:14.365589 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2e7f6-e38f-4df0-af48-dced339b8ff8 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:14.425587 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67317362-99b8-442e-9581-f074efa4ee5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:14.426240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317362-99b8-442e-9581-f074efa4ee5b service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:14.426521 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317362-99b8-442e-9581-f074efa4ee5b service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:14.426797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317362-99b8-442e-9581-f074efa4ee5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:14.429192 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-67317362-99b8-442e-9581-f074efa4ee5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:14.490453 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67317362-99b8-442e-9581-f074efa4ee5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:14.492373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67317362-99b8-442e-9581-f074efa4ee5b service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:14.501953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1417/2843] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:14.958484 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:14.960655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:14.989147 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:15.027459 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8375ee-7db4-433a-b9f6-8523117c2cbd admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:15.031951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:15.032821 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.033140 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.033420 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] PATH_INFO: `/v3/users/9333d2fc707140cf99870047c9011eb3` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.034847 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9333d2fc707140cf99870047c9011eb3)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:15.036781 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:15.083492 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9333d2fc707140cf99870047c9011eb3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.084154 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9333d2fc707140cf99870047c9011eb3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.084766 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9333d2fc707140cf99870047c9011eb3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.091607 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9333d2fc707140cf99870047c9011eb3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.103702 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9333d2fc707140cf99870047c9011eb3'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.116553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1427/2844] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:14 2022] DELETE /identity/v3/users/9333d2fc707140cf99870047c9011eb3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:15.121223 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.121511 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.121785 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.138512 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:15.139691 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:15.197302 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1418/2845] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:15.206903 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:15.208383 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:15.266097 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187c84-9987-4c48-b235-0078d79836a6 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:15.269769 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104f77d-7265-4315-a400-628c321d51fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:15.270453 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104f77d-7265-4315-a400-628c321d51fd service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.270777 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104f77d-7265-4315-a400-628c321d51fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.271058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104f77d-7265-4315-a400-628c321d51fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.280447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104f77d-7265-4315-a400-628c321d51fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:15.281876 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104f77d-7265-4315-a400-628c321d51fd service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:15.292048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1428/2846] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 15:11:15.584904 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:15.587309 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:15.605457 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe8a905-ee04-4041-912e-fc7986e3c225 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:15.611100 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:15.611977 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.612297 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.612530 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] PATH_INFO: `/v3/projects/784e4d4b1a4d4bc9ac4f016a57ec2cfd` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.613875 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=784e4d4b1a4d4bc9ac4f016a57ec2cfd)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:15.615963 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:15.660136 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:15.664524 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784e4d4b1a4d4bc9ac4f016a57ec2cfd', 'request_id': 'req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.670940 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784e4d4b1a4d4bc9ac4f016a57ec2cfd', 'request_id': 'req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.678270 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784e4d4b1a4d4bc9ac4f016a57ec2cfd', 'request_id': 'req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.689771 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] The token cache is being invalidate because project 784e4d4b1a4d4bc9ac4f016a57ec2cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:15.690060 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.690740 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-f803c695-b4dd-4666-b2fa-55d4c6ca3f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:15.692276 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1419/2847] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:15 2022] DELETE /identity/v3/projects/784e4d4b1a4d4bc9ac4f016a57ec2cfd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:15.697326 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75ece2-38bc-4063-8613-351b5271c497 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.697621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75ece2-38bc-4063-8613-351b5271c497 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.697930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75ece2-38bc-4063-8613-351b5271c497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.715795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-fe75ece2-38bc-4063-8613-351b5271c497 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:15.717034 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-fe75ece2-38bc-4063-8613-351b5271c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:15.759125 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:15.777879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1429/2848] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:11:15.785627 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409185a9-c0e9-4ca0-9fee-2180dbd41642 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.785917 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409185a9-c0e9-4ca0-9fee-2180dbd41642 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.786196 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409185a9-c0e9-4ca0-9fee-2180dbd41642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.804341 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-409185a9-c0e9-4ca0-9fee-2180dbd41642 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:15.805581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-409185a9-c0e9-4ca0-9fee-2180dbd41642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:15.815446 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1420/2849] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:15.820200 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75ece2-38bc-4063-8613-351b5271c497 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:15.833516 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75ece2-38bc-4063-8613-351b5271c497 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:15.838622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:15.839286 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.839689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.839854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.840627 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86694) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:11:15.840972 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:15.842572 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:15.854238 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1430/2850] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:11:15.858662 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409185a9-c0e9-4ca0-9fee-2180dbd41642 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:15.867933 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409185a9-c0e9-4ca0-9fee-2180dbd41642 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:15.872685 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:15.873376 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.873666 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.873948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] PATH_INFO: `/v3/projects` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.874677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:15.876581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:15.909253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1421/2851] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:11:15 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:15.913289 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:15.925166 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4142b2-200f-41af-b457-e6294e3189d2 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:15.987707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:15.988388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:15.988701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:15.989048 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] PATH_INFO: `/v3/users` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:15.989751 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:15.991879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:16.015270 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1431/2852] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:11:15 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:16.016485 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:16.027859 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa232bf0-ef50-4fcd-b7b2-5d1f85679012 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:16.032998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:16.033679 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:16.034217 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:16.034494 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:16.035226 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:16.036816 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:16.050291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1422/2853] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:16.051736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:16.062068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8f905-0b86-4dab-b2a7-51609ed93012 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:16.071336 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c893b67-5283-47ca-8740-a917a622d1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:16.072005 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c893b67-5283-47ca-8740-a917a622d1ea admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:16.072293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c893b67-5283-47ca-8740-a917a622d1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:16.072567 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c893b67-5283-47ca-8740-a917a622d1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:16.073345 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c893b67-5283-47ca-8740-a917a622d1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:16.074933 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c893b67-5283-47ca-8740-a917a622d1ea admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:16.085260 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1432/2854] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:16.089406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:16.099520 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec358af-3cca-4613-9fbb-eda2254018d8 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:16.104614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:16.105565 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:16.105854 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:16.106130 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] PATH_INFO: `/v3/projects/4da5b37c95a342eaaa1aee173d381701/users/b952610adaa34e1b8c4d23433bd1558e/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:16.122414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da5b37c95a342eaaa1aee173d381701, user_id=b952610adaa34e1b8c4d23433bd1558e, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:16.125131 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:16.138201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1423/2855] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:11:16 2022] PUT /identity/v3/projects/4da5b37c95a342eaaa1aee173d381701/users/b952610adaa34e1b8c4d23433bd1558e/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:16.143217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f05cf1-71e1-4c77-8318-ca866f7f9a3f None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:16.143513 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f05cf1-71e1-4c77-8318-ca866f7f9a3f None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:16.143803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f05cf1-71e1-4c77-8318-ca866f7f9a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:16.161687 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-a3f05cf1-71e1-4c77-8318-ca866f7f9a3f None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:16.163372 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-a3f05cf1-71e1-4c77-8318-ca866f7f9a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:16.221042 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1433/2856] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:16.231034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:16.232748 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:16.289392 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd3a31-6033-44d8-bcfc-2f9f92a7c7fc admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:16.293103 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:16.293761 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:16.294042 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:16.294318 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:16.305166 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:16.306702 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:16.316098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1424/2857] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:16.925458 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd68a6-e25e-4b39-a922-2c124e43f96d None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:16.925750 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd68a6-e25e-4b39-a922-2c124e43f96d None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:16.926030 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd68a6-e25e-4b39-a922-2c124e43f96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:16.951879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-33fd68a6-e25e-4b39-a922-2c124e43f96d None None] MFA Rules not processed for user `b952610adaa34e1b8c4d23433bd1558e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:16.953115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-33fd68a6-e25e-4b39-a922-2c124e43f96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:16.996634 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:17.022697 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1434/2858] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:17.033181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:17.034855 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:17.097237 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06aa5a-d749-4f88-b190-323b72c1bac5 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:17.101387 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:17.102039 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:17.102322 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:17.102595 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:17.112529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:17.115101 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:17.124951 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1425/2859] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:18.474151 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:18.474743 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:18.475447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:18.498276 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] MFA Rules not processed for user `b952610adaa34e1b8c4d23433bd1558e`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:18.499486 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:18.506468 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1435/2860] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:18.861528 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:18.871339 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082b81-2ac0-4d6f-a27f-1e940284fc6e service cinder] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:18.875316 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7c767-0470-4b5b-aeaa-63357a301de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:18.875953 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7c767-0470-4b5b-aeaa-63357a301de9 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:18.876236 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7c767-0470-4b5b-aeaa-63357a301de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:18.876518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7c767-0470-4b5b-aeaa-63357a301de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:18.883877 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5db7c767-0470-4b5b-aeaa-63357a301de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:18.894212 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7c767-0470-4b5b-aeaa-63357a301de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:18.895644 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7c767-0470-4b5b-aeaa-63357a301de9 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:18.904375 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1426/2861] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:19.364135 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:19.366192 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:19.382896 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ca3e2-5b9f-4e99-8685-387a3e7b1496 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:19.387888 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:19.388913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:19.389293 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:19.389662 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] PATH_INFO: `/v3/users/ba946c8b356345f18f57156b0f047916` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:19.390988 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba946c8b356345f18f57156b0f047916)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:19.393068 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:19.444421 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba946c8b356345f18f57156b0f047916'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:19.445494 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba946c8b356345f18f57156b0f047916'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:19.445767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba946c8b356345f18f57156b0f047916'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:19.458311 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba946c8b356345f18f57156b0f047916'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:19.473622 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba946c8b356345f18f57156b0f047916'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:19.487340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1436/2862] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:19 2022] DELETE /identity/v3/users/ba946c8b356345f18f57156b0f047916 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:19.492340 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:19.492633 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:19.492959 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:19.498675 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:19.514306 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:19.515932 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:19.577371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1427/2863] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:19.587212 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:19.588637 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:19.648493 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3838b-20f8-4fab-8269-a5600fac0e4d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:19.652319 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:19.653117 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:19.653473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:19.653981 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:19.663752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:19.665325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:19.675186 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1437/2864] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:19.842458 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:19.843915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:19.929350 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880f7aa-d2f1-4cca-9a8d-3eca4396da5e None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:19.935040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:19.935824 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:19.936191 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:19.936535 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:19.938489 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:19.944136 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:19.945707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:19.951032 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:19.952634 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:19.964615 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1428/2865] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:19.966338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad770fe-886a-4548-91e6-116a2e9260f7 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:19.980019 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:19.980956 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:19.981245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:19.981542 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] PATH_INFO: `/v3/projects/4b951748299c45f79056a092c4a1153d` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:19.982913 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b951748299c45f79056a092c4a1153d)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:19.984941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:20.030689 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:20.034346 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b951748299c45f79056a092c4a1153d', 'request_id': 'req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.041630 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b951748299c45f79056a092c4a1153d', 'request_id': 'req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.049181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b951748299c45f79056a092c4a1153d', 'request_id': 'req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.061767 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] The token cache is being invalidate because project 4b951748299c45f79056a092c4a1153d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:20.061997 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.062663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.064812 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1438/2866] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:19 2022] DELETE /identity/v3/projects/4b951748299c45f79056a092c4a1153d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:11:20.070098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:20.081187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540faac-8064-4cd2-bf14-6690138693f6 service placement] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:20.140625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:20.141663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:20.141949 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:20.142227 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] PATH_INFO: `/v3/users/4bdb089e6ac241c5aa55d660ac646312` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:20.143518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdb089e6ac241c5aa55d660ac646312)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:20.145569 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:20.188507 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:20.200514 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdb089e6ac241c5aa55d660ac646312'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.201384 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdb089e6ac241c5aa55d660ac646312'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.209597 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdb089e6ac241c5aa55d660ac646312'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.210361 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdb089e6ac241c5aa55d660ac646312'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.219103 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-df286449-240a-4abf-a898-7976cb41853d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdb089e6ac241c5aa55d660ac646312'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.245780 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1429/2867] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:20 2022] DELETE /identity/v3/users/4bdb089e6ac241c5aa55d660ac646312 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:20.574003 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:20.575594 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:20.590122 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd02e9-73aa-4d7e-ba53-9f569767dbfa admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:20.648349 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:20.649257 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:20.649549 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:20.649833 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] PATH_INFO: `/v3/projects/19a3992aafe448e8be21098e46ecf751` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:20.651016 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a3992aafe448e8be21098e46ecf751)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:20.652980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:20.698004 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a3992aafe448e8be21098e46ecf751', 'request_id': 'req-17f1a713-1173-447f-997e-c569db144158'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.703497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a3992aafe448e8be21098e46ecf751', 'request_id': 'req-17f1a713-1173-447f-997e-c569db144158'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.710935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a3992aafe448e8be21098e46ecf751', 'request_id': 'req-17f1a713-1173-447f-997e-c569db144158'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.721736 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] The token cache is being invalidate because project 19a3992aafe448e8be21098e46ecf751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:20.722033 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.722681 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-17f1a713-1173-447f-997e-c569db144158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:20.724210 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1439/2868] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:20 2022] DELETE /identity/v3/projects/19a3992aafe448e8be21098e46ecf751 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:11:20.730313 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ac786-e09e-46cd-9463-09b7014b99c8 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:20.730604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ac786-e09e-46cd-9463-09b7014b99c8 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:20.730878 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ac786-e09e-46cd-9463-09b7014b99c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:20.747845 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-d45ac786-e09e-46cd-9463-09b7014b99c8 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:20.749056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d45ac786-e09e-46cd-9463-09b7014b99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:20.816255 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1430/2869] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:20.822531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441c143-6f95-49fc-8e8c-c880bf6f01f0 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:20.825449 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441c143-6f95-49fc-8e8c-c880bf6f01f0 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:20.825824 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441c143-6f95-49fc-8e8c-c880bf6f01f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:20.846676 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-1441c143-6f95-49fc-8e8c-c880bf6f01f0 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:20.847912 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-1441c143-6f95-49fc-8e8c-c880bf6f01f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:20.859864 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1440/2870] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:11:20.861278 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ac786-e09e-46cd-9463-09b7014b99c8 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:20.871658 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ac786-e09e-46cd-9463-09b7014b99c8 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:20.876533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:20.877298 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:20.877797 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:20.878287 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:20.879209 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:11:20.879722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:20.882080 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:20.899170 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1431/2871] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:11:20.900288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441c143-6f95-49fc-8e8c-c880bf6f01f0 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:20.909663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441c143-6f95-49fc-8e8c-c880bf6f01f0 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:20.914473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:20.915141 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:20.915429 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:20.915709 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:20.916426 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:20.918438 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:20.946001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1441/2872] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:11:20 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:20.949986 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:20.961028 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4e91e-3c1e-4a53-8257-750f2d892d09 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:21.023525 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:21.024202 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:21.024483 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:21.024755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:21.025488 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:21.027407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:21.054874 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1432/2873] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:11:20 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:21.056531 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:21.067359 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa6836-4bb1-44b9-b40b-f648d09371bb admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:21.073023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:21.073775 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:21.074054 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:21.074335 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:21.075065 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:21.076808 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:21.087562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1442/2874] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 15:11:21.091678 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:21.102050 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d227135-2103-4c86-b242-087b73570aaa admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:21.106774 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97bce6-4ab8-4a7e-9309-078c3ab87f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:21.107433 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97bce6-4ab8-4a7e-9309-078c3ab87f46 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:21.107722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97bce6-4ab8-4a7e-9309-078c3ab87f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:21.107998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97bce6-4ab8-4a7e-9309-078c3ab87f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:21.108726 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97bce6-4ab8-4a7e-9309-078c3ab87f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:21.110359 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97bce6-4ab8-4a7e-9309-078c3ab87f46 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:21.121505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1433/2875] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:21.125473 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:21.136617 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223f219-7c55-479a-ae76-8dfb08f1123d admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:21.141324 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:21.142239 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:21.142517 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:21.142788 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] PATH_INFO: `/v3/projects/ad0f035a7d9748b7969080b36cc9c1e7/users/3da712ad180f4efea24e5d83da999bdc/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:21.159181 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0f035a7d9748b7969080b36cc9c1e7, user_id=3da712ad180f4efea24e5d83da999bdc, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:21.161923 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:21.170827 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:21.177265 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1443/2876] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:11:21 2022] PUT /identity/v3/projects/ad0f035a7d9748b7969080b36cc9c1e7/users/3da712ad180f4efea24e5d83da999bdc/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:21.190628 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eeaa7-79d2-4cc2-8e2f-f93dc7525d8c None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:21.190915 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eeaa7-79d2-4cc2-8e2f-f93dc7525d8c None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:21.191394 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eeaa7-79d2-4cc2-8e2f-f93dc7525d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:21.218659 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-273eeaa7-79d2-4cc2-8e2f-f93dc7525d8c None None] MFA Rules not processed for user `3da712ad180f4efea24e5d83da999bdc`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:21.219866 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-273eeaa7-79d2-4cc2-8e2f-f93dc7525d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:21.225515 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:21.293715 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1434/2877] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:11:21.305501 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:21.307220 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:21.367333 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c78b06-6718-4fac-ad93-a7c15521cba3 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:21.371698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212b2c7-b728-4b53-af95-73974c9bc958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:21.372491 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b2c7-b728-4b53-af95-73974c9bc958 service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:21.372733 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b2c7-b728-4b53-af95-73974c9bc958 service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:21.373153 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b2c7-b728-4b53-af95-73974c9bc958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:21.384568 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212b2c7-b728-4b53-af95-73974c9bc958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:21.386558 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212b2c7-b728-4b53-af95-73974c9bc958 service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:21.396895 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1444/2878] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:30.062717 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eeaa7-79d2-4cc2-8e2f-f93dc7525d8c None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:30.135592 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eeaa7-79d2-4cc2-8e2f-f93dc7525d8c None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:30.140090 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:30.140982 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.141260 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.141527 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] PATH_INFO: `/v3/users/3da712ad180f4efea24e5d83da999bdc` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.142712 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3da712ad180f4efea24e5d83da999bdc)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:30.144619 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:30.195955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3da712ad180f4efea24e5d83da999bdc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.195955 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3da712ad180f4efea24e5d83da999bdc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.200317 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3da712ad180f4efea24e5d83da999bdc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.200997 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3da712ad180f4efea24e5d83da999bdc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.208382 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3da712ad180f4efea24e5d83da999bdc'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.225734 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1435/2879] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:30 2022] DELETE /identity/v3/users/3da712ad180f4efea24e5d83da999bdc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:11:30.230183 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.230690 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.230803 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.247964 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:30.249199 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:30.306710 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1445/2880] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:30.316303 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:30.317947 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:30.350505 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:30.379790 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a19208-24d2-4104-b01d-c9fa4dd93417 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:30.384378 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145519f4-2a18-4d0f-839b-3d0675e3ea4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:30.385065 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145519f4-2a18-4d0f-839b-3d0675e3ea4a service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.385351 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145519f4-2a18-4d0f-839b-3d0675e3ea4a service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.385625 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145519f4-2a18-4d0f-839b-3d0675e3ea4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.395102 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145519f4-2a18-4d0f-839b-3d0675e3ea4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:30.396566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145519f4-2a18-4d0f-839b-3d0675e3ea4a service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:30.406570 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1436/2881] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:30.680714 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:30.682463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:30.700023 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15439a5b-505a-4790-bbb5-8311f5db889b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:30.704939 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:30.705831 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.706115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.706389 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] PATH_INFO: `/v3/projects/ad0f035a7d9748b7969080b36cc9c1e7` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.707638 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0f035a7d9748b7969080b36cc9c1e7)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:30.710190 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:30.765927 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0f035a7d9748b7969080b36cc9c1e7', 'request_id': 'req-5e92353f-41b0-4644-9885-3c90bdeb1625'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.771949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0f035a7d9748b7969080b36cc9c1e7', 'request_id': 'req-5e92353f-41b0-4644-9885-3c90bdeb1625'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.780507 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0f035a7d9748b7969080b36cc9c1e7', 'request_id': 'req-5e92353f-41b0-4644-9885-3c90bdeb1625'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.792869 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] The token cache is being invalidate because project ad0f035a7d9748b7969080b36cc9c1e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:30.793160 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.793825 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-5e92353f-41b0-4644-9885-3c90bdeb1625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:30.795387 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1446/2882] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:30 2022] DELETE /identity/v3/projects/ad0f035a7d9748b7969080b36cc9c1e7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:30.800371 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479cb6-51e4-4b1d-869e-73d1879f23a7 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.800729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479cb6-51e4-4b1d-869e-73d1879f23a7 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.800948 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28479cb6-51e4-4b1d-869e-73d1879f23a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.819609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-28479cb6-51e4-4b1d-869e-73d1879f23a7 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:30.820807 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-28479cb6-51e4-4b1d-869e-73d1879f23a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:30.878922 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1437/2883] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:11:30.887347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759d7a-ac39-4971-8801-652502e0e0e5 None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.887467 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759d7a-ac39-4971-8801-652502e0e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.887730 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759d7a-ac39-4971-8801-652502e0e0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.909649 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-be759d7a-ac39-4971-8801-652502e0e0e5 None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:30.910874 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-be759d7a-ac39-4971-8801-652502e0e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:30.918930 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1447/2884] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:30.923289 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28479cb6-51e4-4b1d-869e-73d1879f23a7 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:30.933743 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28479cb6-51e4-4b1d-869e-73d1879f23a7 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:30.938454 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:30.939118 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.941301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.941301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] PATH_INFO: `/v3/domains` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.941301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86693) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:11:30.941301 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:30.942338 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:30.958279 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1438/2885] 10.222.0.72 () {58 vars in 1152 bytes} [Thu Jun 16 15:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:11:30.959428 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be759d7a-ac39-4971-8801-652502e0e0e5 None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:30.968688 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be759d7a-ac39-4971-8801-652502e0e0e5 None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:30.973439 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:30.974102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:30.974388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:30.974698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] PATH_INFO: `/v3/projects` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:30.975390 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:30.977337 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:30.997816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:31.006995 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1448/2886] 10.222.0.72 () {60 vars in 1152 bytes} [Thu Jun 16 15:11:30 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:11:31.011132 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:31.021258 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647bc3c-26cb-463f-9cf6-58dbf968d420 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:31.081657 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:31.082332 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.082614 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.082889 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] PATH_INFO: `/v3/users` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.083575 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:31.085518 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:31.104598 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1439/2887] 10.222.0.72 () {60 vars in 1143 bytes} [Thu Jun 16 15:11:31 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:11:31.108684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:31.119369 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89020ee8-98ea-4f63-b713-50192383672c admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:31.123966 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:31.124626 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.124949 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.125232 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.125962 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:31.127562 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:31.139124 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1449/2888] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:31.142909 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:31.152573 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134b55d-46b5-4889-86d2-780b4759fde9 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:31.157077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036611f2-affc-4199-9aa1-0a3e3bf45013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:31.157719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036611f2-affc-4199-9aa1-0a3e3bf45013 admin admin] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.157998 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036611f2-affc-4199-9aa1-0a3e3bf45013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.158272 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036611f2-affc-4199-9aa1-0a3e3bf45013 admin admin] PATH_INFO: `/v3/roles` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.159240 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036611f2-affc-4199-9aa1-0a3e3bf45013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:31.160796 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036611f2-affc-4199-9aa1-0a3e3bf45013 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:31.170873 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1440/2889] 10.222.0.72 () {58 vars in 1121 bytes} [Thu Jun 16 15:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 15:11:31.174756 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:31.185243 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cbd84-d907-4e64-b5af-eeec46701cd2 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:31.189873 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:31.190778 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.191155 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.191399 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] PATH_INFO: `/v3/projects/78be087dd91449d9943931036c046215/users/851a510030564fc8b0ebba04c0643a2d/roles/b724d12e8d154fc4b71baaf2741624d8` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.207144 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78be087dd91449d9943931036c046215, user_id=851a510030564fc8b0ebba04c0643a2d, role_id=b724d12e8d154fc4b71baaf2741624d8)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:31.209941 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:31.223391 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1450/2890] 10.222.0.72 () {60 vars in 1482 bytes} [Thu Jun 16 15:11:31 2022] PUT /identity/v3/projects/78be087dd91449d9943931036c046215/users/851a510030564fc8b0ebba04c0643a2d/roles/b724d12e8d154fc4b71baaf2741624d8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:11:31.234291 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd9c43-0553-482c-b130-b9d9bd044a4d None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.234586 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd9c43-0553-482c-b130-b9d9bd044a4d None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.235052 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd9c43-0553-482c-b130-b9d9bd044a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.263704 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-88dd9c43-0553-482c-b130-b9d9bd044a4d None None] MFA Rules not processed for user `851a510030564fc8b0ebba04c0643a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:31.264979 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-88dd9c43-0553-482c-b130-b9d9bd044a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:31.290239 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:31.333414 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1441/2891] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 15:11:31.344760 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:31.346533 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:31.406551 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74441c6e-21e9-4de5-98ee-11adc018a3d1 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:31.410495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70b000-4554-42d5-b8af-304cf0e4e08f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:31.411149 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70b000-4554-42d5-b8af-304cf0e4e08f service cinder] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.411433 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70b000-4554-42d5-b8af-304cf0e4e08f service cinder] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.411712 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70b000-4554-42d5-b8af-304cf0e4e08f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.420742 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70b000-4554-42d5-b8af-304cf0e4e08f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:31.422219 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70b000-4554-42d5-b8af-304cf0e4e08f service cinder] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:31.431603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1451/2892] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:31.457319 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd9c43-0553-482c-b130-b9d9bd044a4d None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:31.533124 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd9c43-0553-482c-b130-b9d9bd044a4d None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:31.538533 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:31.539407 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.539803 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.540077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] PATH_INFO: `/v3/users/851a510030564fc8b0ebba04c0643a2d` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.541396 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] RBAC: Authorizing `identity:delete_user(user_id=851a510030564fc8b0ebba04c0643a2d)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:31.543296 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:31.592178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851a510030564fc8b0ebba04c0643a2d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:31.593140 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851a510030564fc8b0ebba04c0643a2d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:31.600973 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851a510030564fc8b0ebba04c0643a2d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:31.601576 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851a510030564fc8b0ebba04c0643a2d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:31.609538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851a510030564fc8b0ebba04c0643a2d'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:31.628412 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1442/2893] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:11:31 2022] DELETE /identity/v3/users/851a510030564fc8b0ebba04c0643a2d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:31.632971 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b9363-125c-456f-b894-6250ade0972b None None] REQUEST_METHOD: `POST` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.633816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b9363-125c-456f-b894-6250ade0972b None None] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.633816 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b9363-125c-456f-b894-6250ade0972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.654395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.auth.core [None req-c99b9363-125c-456f-b894-6250ade0972b None None] MFA Rules not processed for user `b8adc1b6aec146cdaed3b9ac9a211c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86694) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:11:31.655763 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c99b9363-125c-456f-b894-6250ade0972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:31.725115 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1452/2894] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:11:31.733211 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:31.734635 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:31.785006 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:31.801383 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5123a-c37b-462a-9fde-245464d9f37b admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:31.806747 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:31.808531 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:31.808962 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:31.809421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:31.825722 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:31.827607 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:31.839941 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1443/2895] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:11:32.175705 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b9363-125c-456f-b894-6250ade0972b None None] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:11:32.177577 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-c99b9363-125c-456f-b894-6250ade0972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:11:32.194946 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b9363-125c-456f-b894-6250ade0972b None None] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:11:32.202818 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:11:32.205706 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:11:32.206001 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:11:32.206288 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] PATH_INFO: `/v3/projects/78be087dd91449d9943931036c046215` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:11:32.208058 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78be087dd91449d9943931036c046215)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:11:32.210663 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:11:32.264278 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78be087dd91449d9943931036c046215', 'request_id': 'req-f71b1303-8f15-4b50-9956-bccb72d8f169'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:32.270085 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78be087dd91449d9943931036c046215', 'request_id': 'req-f71b1303-8f15-4b50-9956-bccb72d8f169'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:32.273008 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:32.278871 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78be087dd91449d9943931036c046215', 'request_id': 'req-f71b1303-8f15-4b50-9956-bccb72d8f169'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:32.291536 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] The token cache is being invalidate because project 78be087dd91449d9943931036c046215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:11:32.291828 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:32.292459 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:11:32.294059 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1453/2896] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:11:32 2022] DELETE /identity/v3/projects/78be087dd91449d9943931036c046215 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:11:41.833604 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=6 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:11:42.289175 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=1,keystone:SELECT=4 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:12:27.400773 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:12:27.402216 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:27.457673 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5e082-1222-4988-8ba5-f8ffdb083d65 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:12:27.506324 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a41fc7-562f-456d-bb05-cc8e8d6de4b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6812dddc3db746b6893f12f1ae0c715d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:12:27.507608 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a41fc7-562f-456d-bb05-cc8e8d6de4b1 service cinder] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:12:27.507893 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a41fc7-562f-456d-bb05-cc8e8d6de4b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:12:27.508172 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a41fc7-562f-456d-bb05-cc8e8d6de4b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:12:27.510026 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-14a41fc7-562f-456d-bb05-cc8e8d6de4b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:27.584077 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a41fc7-562f-456d-bb05-cc8e8d6de4b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:12:27.586043 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a41fc7-562f-456d-bb05-cc8e8d6de4b1 service cinder] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:12:27.597027 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1444/2897] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:12:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:12:30.173603 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:12:30.175049 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:30.232050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b1303-8f15-4b50-9956-bccb72d8f169 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:12:30.235693 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:12:30.236338 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:12:30.236625 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:12:30.236935 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:12:30.238698 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:30.251951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:12:30.253447 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:12:30.264340 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1454/2898] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:12:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:12:37.592040 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=74 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:12:40.259455 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=26 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:12:42.290406 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128920bd-f4cc-4338-a4bf-29edc5e8d3c3 None None] REQUEST_METHOD: `POST` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:12:42.290923 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128920bd-f4cc-4338-a4bf-29edc5e8d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:12:42.291045 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128920bd-f4cc-4338-a4bf-29edc5e8d3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:12:42.307663 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.auth.core [None req-128920bd-f4cc-4338-a4bf-29edc5e8d3c3 None None] MFA Rules not processed for user `b952610adaa34e1b8c4d23433bd1558e`. Rule list: `[]` (Enabled: `True`). {{(pid=86693) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:12:42.308901 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-128920bd-f4cc-4338-a4bf-29edc5e8d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:42.363636 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1445/2899] 10.222.0.72 () {58 vars in 957 bytes} [Thu Jun 16 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 15:12:42.889746 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:12:42.891274 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:42.961405 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663644b-2883-4f7e-ab8e-98dd7e994e11 service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:12:42.966308 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:12:42.966951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:12:42.967236 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:12:42.967537 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:12:42.969325 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:42.981245 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:12:42.982840 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:12:42.993255 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1455/2900] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:12:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:12:45.017950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128920bd-f4cc-4338-a4bf-29edc5e8d3c3 None None] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:12:45.027719 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128920bd-f4cc-4338-a4bf-29edc5e8d3c3 None None] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:12:45.031311 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:12:45.031956 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:12:45.032250 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:12:45.032529 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:12:45.034481 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:12:45.044117 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:12:45.045566 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:12:45.054019 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1446/2901] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:12:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:12:52.988394 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=32 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:12:55.049950 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=31 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:13:47.900114 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:13:47.901658 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:13:47.967905 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db668e79-61e7-4070-935e-2dbf96d5bcb2 service placement] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:13:47.972388 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:13:47.973306 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:13:47.973615 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:13:47.973879 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] PATH_INFO: `/v3/users/b952610adaa34e1b8c4d23433bd1558e` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:13:47.975067 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b952610adaa34e1b8c4d23433bd1558e)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:13:47.977035 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:13:48.022719 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b952610adaa34e1b8c4d23433bd1558e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.023382 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b952610adaa34e1b8c4d23433bd1558e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.023980 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b952610adaa34e1b8c4d23433bd1558e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.030791 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b952610adaa34e1b8c4d23433bd1558e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.043168 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b952610adaa34e1b8c4d23433bd1558e'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.055738 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1456/2902] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:13:47 2022] DELETE /identity/v3/users/b952610adaa34e1b8c4d23433bd1558e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:13:48.224193 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:13:48.240421 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e4389-91c9-4e74-86ac-d8356f3ba920 service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:13:48.299034 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:13:48.299890 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:13:48.300178 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:13:48.300455 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] PATH_INFO: `/v3/projects/4da5b37c95a342eaaa1aee173d381701` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:13:48.301677 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da5b37c95a342eaaa1aee173d381701)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:13:48.303591 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:13:48.348373 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da5b37c95a342eaaa1aee173d381701', 'request_id': 'req-927dfc81-d392-4a99-9877-064b8d8d5fcb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.354399 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da5b37c95a342eaaa1aee173d381701', 'request_id': 'req-927dfc81-d392-4a99-9877-064b8d8d5fcb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.361581 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da5b37c95a342eaaa1aee173d381701', 'request_id': 'req-927dfc81-d392-4a99-9877-064b8d8d5fcb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.372470 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] The token cache is being invalidate because project 4da5b37c95a342eaaa1aee173d381701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86693) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:13:48.372755 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.373397 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:13:48.374903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1447/2903] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:13:48 2022] DELETE /identity/v3/projects/4da5b37c95a342eaaa1aee173d381701 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:13:54.201677 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:13:54.203178 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:13:54.259707 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb44cc-47c5-420a-b1f2-1c487fe8f979 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:13:54.291814 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:13:54.310423 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:13:54.311081 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:13:54.311368 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:13:54.311650 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:13:54.313454 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:13:54.372050 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:13:54.373569 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:13:54.382463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1457/2904] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:13:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:13:58.370091 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:13:59.045439 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:13:59.046863 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:13:59.059450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927dfc81-d392-4a99-9877-064b8d8d5fcb admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:13:59.063055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fc5135327fe41409cc4ba407b004d1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:13:59.063689 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:13:59.063976 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:13:59.064254 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:13:59.066055 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:13:59.128035 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:13:59.129476 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:13:59.139389 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1448/2905] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:14:02.133408 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:14:02.134897 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:14:02.217209 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb62c6-413e-4328-b185-1770af9cae3b service neutron] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:14:02.222409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:14:02.223142 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] REQUEST_METHOD: `GET` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:14:02.223499 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:14:02.224463 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:14:02.225795 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:14:02.242347 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:14:02.244790 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:14:02.256948 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1458/2906] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:14:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:14:05.254098 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:14:05.255538 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:14:05.268537 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f836d-ac71-4c61-ba45-e244478d8fce service neutron] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:14:05.273714 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:14:05.274609 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:14:05.274903 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:14:05.275247 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] PATH_INFO: `/v3/users/0e6fbab71db54b698572bb8a7d033063` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:14:05.276987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e6fbab71db54b698572bb8a7d033063)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:14:05.278987 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:14:05.339203 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e6fbab71db54b698572bb8a7d033063'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.339978 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e6fbab71db54b698572bb8a7d033063'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.347450 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e6fbab71db54b698572bb8a7d033063'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.348187 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e6fbab71db54b698572bb8a7d033063'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.357478 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e6fbab71db54b698572bb8a7d033063'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.376742 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1449/2907] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:14:05 2022] DELETE /identity/v3/users/0e6fbab71db54b698572bb8a7d033063 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:14:05.563841 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:14:05.580553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9a357-f732-4f90-b31f-735c4abb27a7 service placement] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:14:05.642752 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:14:05.643695 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:14:05.643973 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:14:05.644253 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] PATH_INFO: `/v3/projects/3716e192b1194365acb5595c8f92a862` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:14:05.645495 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3716e192b1194365acb5595c8f92a862)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:14:05.647991 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:14:05.673621 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:SELECT=100 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:14:05.697217 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3716e192b1194365acb5595c8f92a862', 'request_id': 'req-159348b7-b97a-41c6-92e6-42b16598aa99'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.702598 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3716e192b1194365acb5595c8f92a862', 'request_id': 'req-159348b7-b97a-41c6-92e6-42b16598aa99'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.709655 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3716e192b1194365acb5595c8f92a862', 'request_id': 'req-159348b7-b97a-41c6-92e6-42b16598aa99'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.720633 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] The token cache is being invalidate because project 3716e192b1194365acb5595c8f92a862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:14:05.720951 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.721588 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.723102 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1459/2908] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:14:05 2022] DELETE /identity/v3/projects/3716e192b1194365acb5595c8f92a862 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:14:05.727225 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:14:05.729253 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:14:05.742502 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b17bd-720b-48f4-abc5-880da8ea0b33 admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:14:05.798022 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:14:05.798894 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:14:05.799181 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:14:05.799459 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] PATH_INFO: `/v3/users/0eefc98bb5a944e19324804fcbe53bbb` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:14:05.800651 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eefc98bb5a944e19324804fcbe53bbb)` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:14:05.802617 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:14:05.808477 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:14:05.847989 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eefc98bb5a944e19324804fcbe53bbb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.848645 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eefc98bb5a944e19324804fcbe53bbb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.856056 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eefc98bb5a944e19324804fcbe53bbb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.856686 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eefc98bb5a944e19324804fcbe53bbb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.864471 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.notifications [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eefc98bb5a944e19324804fcbe53bbb'} {{(pid=86693) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:05.882261 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1450/2909] 10.222.0.72 () {58 vars in 1223 bytes} [Thu Jun 16 15:14:05 2022] DELETE /identity/v3/users/0eefc98bb5a944e19324804fcbe53bbb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:14:06.053515 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Authenticating user token {{(pid=86694) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:14:06.055339 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.fernet_utils [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86694) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:14:06.070497 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159348b7-b97a-41c6-92e6-42b16598aa99 admin admin] Validating token access rules against request {{(pid=86694) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:14:06.127684 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8adc1b6aec146cdaed3b9ac9a211c0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1304e58d14b64959b1e4ffc146293183', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86694) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:14:06.128553 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:14:06.128854 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:14:06.129143 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] PATH_INFO: `/v3/projects/efc03348fea54514a295e5a980912ece` {{(pid=86694) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:14:06.130409 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc03348fea54514a295e5a980912ece)` {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:14:06.132402 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] RBAC: Authorization granted {{(pid=86694) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:14:06.179395 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc03348fea54514a295e5a980912ece', 'request_id': 'req-ab216a8b-f03e-4fdb-8d82-590b891a83ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:06.184783 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc03348fea54514a295e5a980912ece', 'request_id': 'req-ab216a8b-f03e-4fdb-8d82-590b891a83ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:06.191701 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc03348fea54514a295e5a980912ece', 'request_id': 'req-ab216a8b-f03e-4fdb-8d82-590b891a83ca'} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:06.202478 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] The token cache is being invalidate because project efc03348fea54514a295e5a980912ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86694) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 15:14:06.202754 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:06.203384 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG keystone.notifications [None req-ab216a8b-f03e-4fdb-8d82-590b891a83ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86694) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 15:14:06.204937 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: [pid: 86694|app: 0|req: 1460/2910] 10.222.0.72 () {58 vars in 1232 bytes} [Thu Jun 16 15:14:06 2022] DELETE /identity/v3/projects/efc03348fea54514a295e5a980912ece => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:14:15.878736 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:14:16.199462 dv-cinder-837165-3-iscsi devstack@keystone.service[86694]: DEBUG dbcounter [-] [86694] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=86694) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:16:25.761012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Authenticating user token {{(pid=86693) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:16:25.762456 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:16:25.830613 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dce370-a6b5-4f9b-aeec-05f0b1e8481d admin admin] Validating token access rules against request {{(pid=86693) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:16:25.889721 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3530102-93fd-4008-8f44-22cfe70ccaff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '848d858044ed43e69aa055491cc29d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2f11c4ae5736434a9e903b50b7951f1a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86693) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:16:25.890441 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3530102-93fd-4008-8f44-22cfe70ccaff service placement] REQUEST_METHOD: `GET` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:16:25.890729 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3530102-93fd-4008-8f44-22cfe70ccaff service placement] SCRIPT_NAME: `/identity` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:16:25.891012 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3530102-93fd-4008-8f44-22cfe70ccaff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86693) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:16:25.892829 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.fernet_utils [None req-f3530102-93fd-4008-8f44-22cfe70ccaff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86693) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:16:25.904590 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3530102-93fd-4008-8f44-22cfe70ccaff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:16:25.906201 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3530102-93fd-4008-8f44-22cfe70ccaff service placement] RBAC: Authorization granted {{(pid=86693) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:16:25.916879 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: [pid: 86693|app: 0|req: 1451/2911] 10.222.0.72 () {60 vars in 1365 bytes} [Thu Jun 16 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 15:16:35.911776 dv-cinder-837165-3-iscsi devstack@keystone.service[86693]: DEBUG dbcounter [-] [86693] Writing DB stats keystone:SELECT=58 {{(pid=86693) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}